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performance  monitoring,  fault  detection,  fault  localization  design  guidance 

INTRODUCTION 

Early  attempts  at  Performance  Monitoring,  Fault  Detection,  and  Fault 
Localization  (PM/FD/FL)  have  largely  been  hit  or  miss  with  little  consistent 
acceptance  among  the  )Navy  or  contractors  with  definitions  or  technical 
approach . 

/ 

— V'  This  report  is  a  formal  approach  to  standardize  specifications  and 
descriptions  of/,  PM/FD/FI>  f or  all  disciplines,  hardware,  firmware,  software, 
reliability,  maintainability,  configuration  management,  and  integrated  logistic 
support . 

Although  this  report  will  be  most  useful  in  a  new  design  and  designs  that 
use  microprocessors,  as  applicable^ this  report  should  be  useful  in  older 
designs  including  GFE  and  also  in  any  designs  that  employ  electrical  or 
electronic  components.  (.  : 

No  attempts  have  been  made  in  this  report  to  predict,  evaluate,  or  record 
failure  trends  or  other  failure  analysis.  Hopefully,  this  will  be  addressed  in 
the  future. 

Using  the  approach  described  herein  will  help  to  determine  whether  or  not 
PM/FD/FL  designs  are  meeting  design  specification,  performance,  or  contractual 
requirements  both  on  the  systems  (macro  view)  level  and  also  on  the  individual 
SEM  module  (micro)  level.  Testing  and  certification  documentation  for  PM/FD/FL 
is  addressed  in  depth. 

This  report  is  designed  to  help  standardize  and  change  the  design, 
approach,  certification,  and  testing  of  PM/FD/FL  to  a  uniform  engineering 
design.  It  is  divided  according  to  PM/FD/FL  tasks  and  contains  several 
appendixes.  Appendix  A  contains  statement  of  work  samples;  appendix  B  contains 
a  glossary  of  terms.  PM/FD/FL  samples  presentation  slides,  applicable  data 
item  descriptions  (DIDs),  and  sample  contract  data  requirements  lists  (CDRLs) 
are  presented  in  appendixes  C,  D,  and  E,  respectively. 


TR  8315 


PERFORMANCE  MONITORING  TASKS 


TASK  101  PERFORMANCE  MONITORING  PROGRAM  PLAN 

101.1  Overview:  The  Performance  Monitoring  Program  Plan  shall 
be  designed  as  a  basic  tool  to  assist  the  contractor  in 
implementing  an  effective  performance  monitoring  development 
program.  The  Government  shall  also  use  the  plan  to  (1)  evaluate 
the  contractor’s  approach  to,  and  his  execution  of,  performance 
monitoring  tasks;  (2)  evaluate  the  adequacy  of  his  procedures  for 
planning,  implementing,  and  controlling  the  performance 
monitoring  tasks;  and  (3)  evaluate  the  ability  of  his 
organizational  structure  to  focus  on  performance  monitoring 
activities/problems . 

101.2  Purpose:  The  purpose  of  Task  101  is  to  develop  a 

Performance  Monitoring  Program  Plan  that  identifies  and 
integrates  all  program  tasks  necessary  to  accomplish  performance 
monitoring  requirements  of  the  Prime  Item  Development 
Specification  (PIDS)  and  the  Statement  of  Work  (SOW). 

101.3  Task  Description:  The  Performance  Monitoring  Program  Plan 

shall  be  prepared  to  provide,  as  a  minimum,  the  following: 

1.  A  description  of  how  the  performance  monitoring  program 
will  be  conducted  to  meet  the  requirements  of  the  PIDS 
and  the  SOW. 

2.  A  description  of  how  performance  monitoring  interfaces 
with  total  system  design. 

3.  A  detailed  description  of  how  each  specific  performance 
monitoring  functional  failure  requirement  will  be 
performed  or  complied  with. 

4.  The  procedures  to  evaluate  the  status  and  control  of 
each  task  and  identification  of  the  organizational  unit 
with  the  authority  and  responsibility  for  executing  each 
task. 

5.  A  schedule  with  estimated  start  and  completion  points 
for  each  performance  monitoring  program  activity  or 
task. 

6.  The  identification  of  known  performance  monitoring 
problems  to  be  solved,  an  assessment  of  the  impact  of 
these  problems  on  meeting  specified  requirements,  and 
the  proposed  solutions  or  proposed  plan  to  solve  them. 

7.  The  procedure  or  methods  for  recording  the  status  of 
actions  to  resolve  problems. 
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8.  The  designation  of  performance  monitoring  milestones, 
including  design,  review  (PDR,  CDR,  IPR) ,  and  test. 

9.  The  method  by  which  the  performance  monitoring 
requirements  are  disseminated  to  designers  and 
associated  personnel  and  how  design  interfaces  are 
accomplished. 

10.  Identification  of  key  personnel  for  managing  the 
performance  monitoring  program  and  the  level  of 
authority  for  problem  resolution. 

11.  Description  of  the  management  structure,  including 
interrelationship  between  line,  service,  staff,  and 
policy  organizations. 

12.  The  performance  monitoring  design  review  checklist  will  be 
used  to  ensure  that  design  meets  requirements 

When  approved  by  the  Government  the  Performance  Monitoring 
Program  Plan  shall  become  a  basis  for  evaluation  of  contractual 
compliance . 

TASK  102  PERFORMANCE  MONITORING  PROGRAM  DESIGN  REVIEWS 

102.1  Purpose:  The  purpose  of  Task  102  is  to  establish  a 

requirement  for  the  contractor  to  conduct  formal  and  informal 
performance  monitoring  program  design  reviews. 

102.2  Task  Description:  Performance  monitoring  formal  design 

reviews  shall  be  conducted  in  accordance  with  the  requirements  of  MIL- 
STD-1521B  on  a  schedule  approved  by  the  Government  responsible 
agency.  Informal  performance  monitoring  in-process  reviews  shall  be 
conducted  at  least  quarterly  until  formal  Critical  Design  Review 
(CDR)  on  a  schedule  mutually  agreed  upon  by  the  Government 
responsible  agency  and  the  contractor.  The  contractor  proposed 
formal  and  informal  design  review  schedule  shall  be  provided  as  part 
of  the  Performance  Monitoring  Program  Plan. 


-4- 


V4 


>«r 


?WBirm.nwjW.7  v  v>  ,-i» 


TR  8315 

In  addition  to  the  formal  design  review  requirements  of  MIL-STD- 
1521B,  the  following  formal  and  informal  design  reviews  shall 
include  the  performance  monitoring  requirements  indicated  below: 

1.  Preliminary  Design  Review  (PDR): 

a.  Updated  performance  monitoring  program  status, 
including 

(1)  performance  monitoring  modeling; 

(2)  performance  monitoring  allocations; 

(3)  performance  monitoring  predictions; 

(4)  performance  monitoring  compliance  with 
specifications  and 

(5)  design  guideline  criteria. 

b.  Problems  affecting  performance  monitoring. 

c.  Performance  monitoring  critical  items. 

2.  Critical  Design  Review  (CDR): 

a.  Performance  monitoring  compliance  with 
specifications . 

b.  Performance  monitoring  predictions  and  analyses. 

c.  Performance  monitoring  critical  items. 

d.  Problems  affecting  performance  monitoring. 

e.  Identification  of  circuits  where  the  design  requires 
high  reliability  components  and  the  software/ firmware 
employs  an  extra-large  number  of  lines  of  code. 

3.  In-Process  Performance  Monitoring  Reviews( IPR) : 

a.  Discussion  of  those  performance  monitoring  items 
previously  listed  in  Sections  1  and  2. 

b.  Results  of  performance  monitoring  test  analyses. 

c.  Test  schedule:  start  and  completion  dates. 


d.  Performance  monitoring  parts,  design,  reliability, 
and  schedule  problems. 

e.  Status  of  assigned  action  items. 

f.  Contractor's  assessment  of  performance  monitoring 
design  effectiveness. 

g.  Other  topics  and  issues  on  the  agenda  agreed  to  by 
the  contractor  and  the  Government. 

h.  Results  of  applicable  performance  monitoring  growth 
testing . 

4.  Test  Readiness  Review: 

a.  Performance  monitoring  analyses  status  and  primary 
prediction. 

b.  Test  schedule. 

c.  Test  profile. 

d.  Test  plan  including  failure  definition. 

e.  Test  report. 

5.  Production  Readiness  Review:  Results  of  applicable  performance 
monitoring  growth  testing. 

TASK  103  PERFORMANCE  MONITORING  MODELING 

103.1  Overview:  Both  quantitative  and  qualitative  analyses  are 

useful  in  determining  where  performance  monitoring  resources 

should  be  applied.  The  analyses  identify  improvements  that  must  be  made 

if  requirements  are  to  be  met.  In  particular,  the 

analyses  are  efficient  work  direction  tools  because  they  can 

confirm  system  adequacy  or  identify  the  need  for  design  change, 

provided  they  are  accomplished  in  conjunction  with,  or  reviewed 

by,  other  disciplines. 

103.2  Purpose:  The  purpose  of  Task  103  is  to  develop  a 

performance  monitoring  model  for  making  numerical  allocations  and 
estimates  to  evaluate  system/subsystem/equipment  performance 
monitoring  effectiveness. 
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103.3  Task  Description:  A  performance  monitoring  mathematical 

model  based  on  system/ subs yst em/equipment  functions  shall  be 
developed  and  maintained.  As  the  design  evolves,  a  performance 
monitoring  block  diagram  (fault  isolation  groupings)  with 
associated  allocations  and  predictions  for  all  elements  in  the 
FIG  shall  be  created.  The  performance  monitoring  block  diagram 
shall  be  keyed  and  traceable  to  the  functional  block  diagram, 
schematics,  drawings,  and  specifications.  The  model  outputs 
shall  be  expressed  in  terms  of  performance  monitoring 
requirements.  As  changes  occur,  the  model  shall  be  updated  to 
include  hardware  or  software/firmware  design  changes. 

The  performance  monitoring  model  shall  be  updated  with 
information  resulting  from  relevant  tests  and  changes  in  item 
configuration. 


TASK  104  PERFORMANCE  MONITORING  ALLOCATION 


104.1  Overview:  System  performance  monitoring  requirements 

evolve  in  a  number  of  ways,  from  informed  judgments  to  analyses 
based  on  empirical  data.  The  requirements  are  designed  to 
minimize  the  total  cost  of  developing,  procuring,  and  operating 
the  system  during  its  life  cycle.  The  integrity  of  the  system  is 
maintained  by  adequate  top-down  design  that  ensures  the  ability  of  the 
system  to  meet  specified  requirements. 

104.2  Purpose:  The  purpose  of  Task  104  is  to  ensure  that,  once 

quantitative  system  requirements  have  been  determined,  they  are 
allocated  or  apportioned  to  lower  levels. 

104.3  Task  Description:  Both  the  mission  and  mission  integrity 

requirements  shall  be  allocated  to  the  level  specified  and  shall 
be  used  to  establish  the  baseline  requirements  for  designers  and 
software/ firmware  personnel.  Requirements  consistent  with  the 
allocations  shall  be  imposed  on  all  subcontractors  and  suppliers.  The 
allocated  values  shall  be  included  in  appropriate  sections  of  any 
procurement  specifications,  critical  item  specifications,  and  contract  end 
item  specifications  to  subcontractors/suppliers. 

All  allocated  performance  monitoring  "alues  established  by  the 
contractor  and  included  in  subcontract  item  specifications  shall 
be  consistent  with  the  mathematical  model  required  in  Task  103. 
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TASK  105  PERFORMANCE  MONITORING  PREDICTION 


105.1  Overview:  Allocations  are  determined  from  the  system 

performance  monitoring  requirements  to  provide  lower  level 
requirements  which  are  levied  on  the  designers  and  software/ 
firmware  engineers.  As  design  work  progresses,  predictions 
based  on  previously  generated  data  and  assessments  based  on 
program  test  data  are  used  to  determine  whether  the  allocated 
requirement  can  or  will  be  met. 

Predictions  combine  lower  level  performance  monitoring  data  to 
indicate  equipment  performance  monitoring  performance  at 
successively  higher  levels,  from  subassemblies  through  subsystem 
to  system.  Predictions  falling  short  of  requirements  at  any 
level  signal  the  need  for  management  and  technical  attention. 

105.2  Purpose:  The  purpose  of  Task  105  is  to  estimate  the 

performance  monitoring  capability  of  the  system,  subsystem, 
equipment,  hardware,  and  software/firmware  and  to  determine 
whether  or  not  the  performance  monitoring  requirements  can  be 
achieved  with  the  proposed  design. 

105.3  Task  Description:  Performance  monitoring  predictions 

shall  be  made  for  the  system,  subsystem,  equipment,  hardware,  and 
software/firmware.  The  predictions  shall  include  the  probability  of  a 
functional  failure,  the  probability  of  not  diagnosing  a  performance  fault, 
and  the  probability  of  incorrectly  diagnosing  a  performance  fault. 
Predictions  shall  be  made  (1)  to  show  the  ability  of  the  performance 
monitoring  function  to  assess  system  and  subsystem  integrity,  (2)  to 
provide  a  basis  for  life-cycle  and  logistic  support  analyses,  and  (3)  to 
provide  a  basis  for  estimating  system  availability. 

The  predictions  shall  be  made  by  using  the  associated  performance 
monitoring  block  diagram  and  performance  monitoring  coverage  data  and 
shall  be  approved  by  the  Government.  Items  and  equipment  shall  not  be 
excluded  from  the  predictions  for  any  reason. 

TASK  106  PERFORMANCE  MONITORING  FAULT  TREE 

106.1  Overview:  The  performance  monitoring  fault  tree  is  used 

as  a  basic  tool  by  the  contractor,  the  government  program  office,  and  the 
independent  verification  and  validation  (IV&V)  groups  to  determine  the 
path  of  initial  fault  observation  to  the  final  display. 

106.2  Purpose:  Tiie  specific  purpose  of  the  performance  monitoring  fault 
tree  is  to  assist  in  designing,  testing,  and  implementing  an  effective 
performance  monitoring  subprogram.  The  performance  monitoring  fault  tree 
shall  be  used  to  evaluate  the  contractor's  approach  to,  and  confirmation 
of,  adherence  to  PIDS  requirements. 
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106.3  Task  Description:  The  fault  tree  shall  indicate  each 

fault  test  point  and  the  pass/fail  levels  at  that  test  point. 

Each  functional  failure  shall  be  labeled  and  described.  This 
description  shall  include 

1.  All  test  points  that  are  used  to  determine  if  a 
functional  failure  exists.  Where  a  votive  or  count 
determination  (e.g.,  3  out  of  5)  exists,  descriptions 
shall  be  supplied. 

2.  Identification  of  test  points  that  are  common  to  any 
other  PM/FD/FL  subprograms  or  tests. 

3.  The  contractor's  verification  that  determinations  of 
performance  monitoring  faults  to  indicate  a  functional 
failure  are  direct,  not  made  by  inference  or  other  indirect 
observations . 

4.  Proof  that  software/firmware  programs  that  are  used  for 
determination  are  labeled  and  referenced  to  the 
configuration  item  where  they  are  located. 


TASK  107  PERFORMANCE  MONITORING  FUNCTION  CERTIFICATION 

107.1  Overview:  It  is  vital  that  designs  be  tested,  not  only  to  see 

if  the  designs  themselves  are  functional  and  fault  free,  but  also  that  the 
designs  meet  not  only  the  'letter  of  the  specification'  but  also  meet  the 
actual  intent  of  the  specification.  Verification  of  design  to 
specification  should  be  performed  at  all  levels  of  development  and  when  it 
appears  to  have  been  completed,  retesting  and  verification  should  occur, 
starting  at  the  original  design  team,  to  contractor  quality  assurance 
personnel,  to  independent  test  teams,  and  finally  by  the  Government. 

107.2  Purpose:  The  purpose  of  this  task  is  to  verify  and  demonstrate 
that  qualifying  tests  to  show  adherence  to  PIDS  requirements  are  in  enough 
detail,  quality,  frequency,  and  number  to  provide  a  high  level  of 
confidence. 

107.3  Task  Description:  Task  107  performance  monitoring  function 
certification  is  a  series  of  qualifying  tests  to  determine  adherence  to 
the  PIDS  requirements.  These  tests  shall  be  designed  to  answer,  as  a 
minimum,  the  following  questions: 

1.  Did  the  performance  monitoring  function  detect  the 
fault? 


Did  the  performance  monitoring  function  indicate  the 
proper  operational  status? 

Did  performance  monitoring  provide  effective  fault 
isolation  information  for  corrective  maintenance 
actions? 

Did  performance  monitoring  provide  information  for 
further  tests  that  could  affirm  the  problem? 

Did  the  performance  monitoring  function  provide  infor¬ 
mation  regarding  the  impact  of  the  fault  to  the  system? 
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6.  What  was  the  latency  time  between  the  occurrence  of  the 
fault  and  the  final  indication  on  the  panel? 

7.  Was  there  any  ambiguity  surrounding  the  fault  or  the 
correction? 

8.  What  are  the  total  number  of  undetected  faults  in  any 
given  period?  Why  were  they  not  detected? 

9.  What  is  the  latency  time  from  software/hardware  fault 
to  automatic  rebooting? 

10.  What  is  the  latency  time  to  detect  a  problem  in  the 
computer  firmware/hardware  that  is  not  correctable  by 
automatic  rebooting? 


TASK  108  PERFORMANCE  MONITORING  INDEPENDENT  VERIFICATION  &  VALIDATION 

108.1  Overview:  Independent  performance  monitoring  verification  and 

validation  performed  by  a  scientific  team  not  involved  in  the  design, 
development,  and  tests  ensures  that  the  performance  monitoring  design 
meets  the  PIDS  requirements.  The  independent  IV&V  team  will  ensure  that 
the  performance  monitoring  subprogram  will  not  fail  and  will  perform  to 
its  intended  capacity. 

108.2  Purpose:  The  purpose  of  Task  108  is  to  independently 

determine  that  the  PIDS  and  SOW  requirements  have  been  met. 

108.3  Task  Description:  Procedures  shall  be  independently 

established,  maintained,  and  implemented,  to  be  performed  by  test  and 
analysis,  to  verify  and  validate  the  ability  of  the  performance  monitoring 
subsystem  to  meet  all  of  the  PIDS  and  SOW  requirements.  The  functional 
testing  of  the  design  shall  employ  methodologies  of  great  stress  and 
strain  to  the  hardware  and  firmware/software. 

The  performance  monitoring  subsystem  shall  be  tested  under  worst-case 
actual  operational  conditions.  The  documentation  produced  by  the  IV&V 
team  shall  include  but  not  be  limited  to 

1.  The  test  plan  for  the  tests  that  will  be  conducted, 
including  the  operational  conditions  under  which  the 
tests  will  be  performed. 

2.  The  actual  test  procedures  with  dates,  test  engineer, 
location,  and  all  other  pertinent  information. 
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3.  Identification,  description,  listings,  and  source  code 
for  IV&V  test  programs . 

4.  Complete  test  reports,  results,  deficiencies,  problems, 
and  observations. 

TASK  109  PERFORMANCE  MONITORING  CONFIGURATION  MANAGEMENT  OVERVIEW 

109.1  Overview:  Separate  plans  and  procedures  shall  be  prepared  and 
adopted  for  the  collecting,  cataloging,  and  describing,  of  all  designs, 
changes,  implementations,  problems,  programs,  test  procedures,  test 
results,  test  findings,  conclusions,  and  observations  for  the  performance 
monitoring  program,  subprogram,  elements,  hardware  firmware/software. 

109.2  Purpose:  The  purpose  of  this  task  is  to  verify  and  demonstrate  the 
configuration  management  specifications,  detail,  quantity,  quality,  and 
media  are  sufficient  to  meet  the  requirements  for  the  program. 

109.3  Task  Description:  The  contractor  shall  use  a  configuration 

management  program  for  the  performance  monitoring  system,  subsystem, 
program  elements,  hardware,  software/firmware.  Enginering  Change 
Proposals  (ECPs),  PDRs,  listings  of  PIDs  requirements  (as  interpreted  by 
the  contractor),  and  any/all  other  documentation  pertinent  to  the 
performance  monitoring  system.  Data  shall  also  include  but  not  be  limited 
to 

1.  Requirements  as  provided  to  subcontractors. 

2.  Subcontractors  response  and  interpretation  of 
requirements , 

3.  Test  procedures  by  contractor  and  subcontractors. 

4.  Any  qualification  tests,  results,  conclusions, 
observations . 

5.  Changes  as  provided  by  the  program  office,  as  initiated 
by  the  contractor,  as  required  by  the  results  from  new 
data,  as  required  for  any  other  purposes. 

6.  All  data  item  requirements. 

7.  All  data  necessary  for  life  cycle  support,  test, 
certification. 

8.  Design  drawings,  source  code,  program  language(s),  and 
other  documentation  to  provide  the  capability  for 
independent  certification,  duplication  of  the  system, 
subsystem,  elements,  firmware/software,  and  hardware. 


TASK  110  PERFORMANCE  MONITORING  FAULT  IMPACT 


110.1  Overview:  Not  all  faults  have  the  same  effect  on  system 
integrity,  system  effectiveness,  or  system  operational  availability.  Some 
faults  mask  others  that  may  have  more  of  an  impact  on  system  integrity. 
Similarly,  certain  portions  of  systems  have  redundancies,  either  natural 
or  planned.  In  the  case  of  faults  in  redundant  portions,  it  may  be 
possible  to  schedule  maintenance  for  some  planned  time.  The  faults,  then, 
are  not  critical  to  system  integrity  or  operations,  provided  they  are 
recorded  and  repaired  at  the  next  repair  cycle  time.  When  a  multitude  of 
faults  occur,  there  are  often  one  or  two  major  faults  that  have  had  a 
ripple  effect  and  cause  other  faults  to  occur.  The  ripple  impact  is 
potentially  dangerous  because  the  impact  on  system  operation  will  not  be 
easily  determined  and  the  parent  fault(s)  of  the  problem  may  not  be 
identified.  By  assigning  levels  of  impact  to  each  fault,  there  is  a 
better  probability  of  correctly  assessing  the  fault  impact,  determining 
system  impact,  and  looking  for  the  most  damaging  fault  first. 

In  effect,  giving  a  level  of  impact  to  each  fault  allows  for  more  correct 
diagnosis  of  the  actual  cause  of  failures.  For  example,  if  a  power  supply 
were  to  be  in  fault,  most  of  the  units  that  had  test  points  for  the 
performance  monitoring  subsystem  would  give  indication  of  failure.  For 
this  reason,  given  the  multitude  of  possible  faults  occurring  or  seeming 
to  occur  all  at  once,  it  is  necessary  to  determine  the  impact  of  every 
test  point  used  for  the  performance  monitoring  subsystem.  The  standard 
procedure  is  to  give  each  fault  an  impact  level  (sometimes  called  a 
priority  level). 

110.2  Purpose:  The  purpose  of  Task  110  is  to  test  the  ability 

of  the  performance  monitoring  subsystem  to  correctly  determine  the  impact 
of  faults  that  it  has  detected  with  respect  to  the  integrity  and 
effectiveness  of  the  major  system.  Additionally,  this  task  is  to 
demonstrate  that  faults  do  not  mask  each  other  when  they  occur  at  the  same 
time.  This  task  is  also  to  demonstrate  that  the  fault  determin-  ation 
will  allow  formaintenance  actions  in  the  required  time  and  to  the  proper 
fault  isolation  group. 

110.3  Task  Description:  A  performance  monitoring  plan  for  fault  impact 

shall  be  developed  and  include  but  not  be  limited  to 

1.  A  description  of  how  fault  impact  is  handled  by  the 
system. 

2.  A  description  of  how  the  performance  monitoring  design 
meets  the  PIDS  requirements. 
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3.  A  test  plan  and  procedure  for  testing  fault  impact. 

4.  A  worst-case  series  of  tests  and  their  evaluations 
regarding  which  fault  created  the  problem. 

5.  Test  cases  intended  to  be  ambiguous  with  respect  to 
which  fault  initiated  the  problem. 

6.  Stress  test  cases  under  actual  operating  conditions. 

7.  A  listing  of  test  panel  indications  for  all  tests. 

Documentation  of  all  fault  impact  test  results  shall  be  included 
in  this  task. 
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FAULT  DETECTION  TASKS 


TASK  201  FAULT  DETECTION  PROGRAM  PLAN 

201.1  Overview:  The  Fault  Detection  Program  Plan  shall  be  designed  as  a 
basic  tool  to  assist  the  contractor  in  implementing  an  effective  fault 
detection  development  program.  The  government  will  also  use  the  plan  to 
(1)  evaluate  the  contractor's  approach  to,  and  his  execution  of,  fault 
detection  tasks,  (2)  evaluate  the  adequacy  of  his  procedures  for 
planning,  implementing,  and  controlling  the  fault  detection  tasks,  and 
(3)  evaluate  the  ability  of  his  organizational  structure  to  focus  on 
fault  detection  activities/problems. 

201.2  Purpose:  The  purpose  of  Task  201  is  to  develop  a  Fault  Detection 
Program  Plan  that  identifies  and  integrates  all  program  tasks  necessary 
to  accomplish  the  requirements  of  the  Prime  Item  Development 
Specification  (PIDS)  and  the  Statement  of  Work  (SOW). 

201.3  Task  Description:  A  Fault  Detection  Program  Plan  shall  be 
prepared  to  provide,  as  a  minimum,  the  following: 

1.  A  description  of  how  the  fault  detection  program  will  be 
conducted  to  meet  the  requirements  of  the  PIDS  and  the 
SOW. 

2.  A  description  of  how  fault  detection  design  interfaces 
with  total  system  design. 

3.  A  detailed  description  of  how  each  specific  fault 
detection  requirement  will  be  performed  or  complied 
with. 

4.  The  procedures  to  evaluate  the  status  and  control  of 
each  task,  and  identification  of  the  organizational  unit 
with  the  authority  and  responsibility  for  executing  each 
task. 

5.  A  schedule  with  estimated  start  and  completion  points 
for  each  fault  detection  program  activity  or  task. 

6.  The  identification  of  known  fault  detection  problems  to 
be  solved,  an  assessment  of  the  impact  of  these  problems 
on  meeting  specified  requirements,  and  the  proposed 
solutions  or  proposed  plan  to  solve  them. 

7.  The  procedure  or  methods  for  recording  the  status  of 
actions  taken  to  resolve  problems. 


8.  The  designation  of  fault  detection  milestones,  including 
design,  review  (PDR,  CDR,  IPR),  and  test. 
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9.  The  method  by  which  the  fault  detection  requirements  are 
disseminated  to  designers  and  associated  personnel,  and 
how  design  interfaces  are  accomplished. 

10.  Identification  of  key  personnel  for  managing  the  fault 
detection  program  and  the  level  of  authority  for  problem 
resolution. 

11.  Description  of  the  management  structure,  including 
interrelationship  between  line,  service,  staff,  and 
policy  organizations . 

12.  The  fault  detection  design  review  checklist  that  will  be 
used. 

When  approved  by  the  Government,  the  Fault  Detection  Program  Plan 
shall  become,  together  with  the  SOW,  a  basis  for  evaluation  of 
contractual  compliance . 

TASK  202  FAULT  DETECTION  PROGRAM  DESIGN  REVIEWS 

202.1  Overview:  Periodic  design  reviews  should  be  held  to  establish 
whether  or  not  the  projected  design  will  meet  the  requirements  of  the 
specifications.  At  the  onset,  reviews  should  be  held  more  frequently  to 
ensure  that  the  contractor  does  not  proceed  with  unsuitable  designs.  The 
reviews  are  also  to  confirm  that  the  contractor  is  not  only  meeting  the 
'wording'  of  the  specification,  but  also  the  intent  of  the  specification. 
Design  reviews  may  be  held  at  any  time  and  it  is  not  necessary  that  they 
be  separate  from  other  reviews,  providing  that  they  are  given  proper 
emphasis  as  would  be  required  to  ensure  that  the  contractor  is  performing 
and  adhering  to  Government  standards  and  requirements  and  also  to  other 
sections  of  this  entire  specification 

202.2  Purpose:  The  purpose  of  Task  202  is  to  establish  a  requirement 
for  the  contractor  to  conduct  formal  and  informal  fault  detection  program 
design  reviews. 

202.3  Task  Description:  Fault  detection  formal  design  reviews  shall  be 
conducted  in  accordance  with  the  requirements  of  MIL-STD-1521B  or  a 
schedule  approved  by  the  Government.  Informal  in-process  fault  detection 
reviews  shall  be  conducted  at  least  quarterly  until  formal  CDR  on  a 
schedule  mutually  agreed  upon  by  the  Government  and  the  contractor.  The 
contractor-proposed  formal  and  informal  design  review  schedule  shall  be 
provided  as  part  of  the  Fault  Detection  Program  Plan. 


TR  8315 


In  addition  to  the  formal  design  review  requirements  of  MIL-STD-1521B, 
the  following  formal  and  informal  design  reviews  shall  include  review  of 
the  fault  detection  items  indicated  below. 

1.  Preliminary  Design  Review  (PDR): 

a.  Updated  fault  detection  program  status  including 

1)  Fault  detection  modeling; 

2)  Fault  detection  allocation; 

3)  Fault  detection  predictions; 

4)  Fault  detection  compliance  with  specifications; 

5)  Design  guideline  criteria. 

b.  Problems  affecting  fault  detection. 

c.  Fault  detection  critical  items. 

2.  Critical  Design  Review  (CDR): 

a.  Fault  detection  compliance  with  specifications. 

b.  Fault  detection  predictions  and  analyses. 

c.  Fault  detection  critical  items. 

d.  Problems  affecting  fault  detection. 

e.  Identification  of  circuits  where  the  design  requires 
high  reliability  components  and  the  software 
firmware  employs  an  extra- large  number  of  lines  of 
code. 

3.  In-Process  Fault  Detection  Reviews  (IPR): 

a.  Discussion  of  those  fault  detection  items  previously 
listed  under  Sections  a  and  b. 

b.  Results  of  fault  detection  test  analyses. 

c.  Test  schedule:  start  and  completion  dates. 

d.  Fault  detection  parts,  design,  reliability,  and 
schedule  problems. 

e.  Status  of  assigned  action  items. 

f.  Contractor's  assessment  of  fault  detection  design 
effectiveness . 

g.  Other  topics  and  issues  on  the  agenda  agreed  to  by 
the  contractor  and  the  Government. 

h.  Results  of  applicable  fault  detection  growth 
testing. 

4.  Test  Readiness  Review: 

a.  Fault  detection  analyses  status  and  primary 
prediction. 

b.  Test  schedule. 

c.  Tes*-  profile. 

d.  Test  plan  including  failure  definition. 

e.  Test  report. 
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5.  Production  Readiness  Review  results  of  applicable  fault 
detection  growth  testing. 

TASK  203  FAULT  DETECTION  MODELING 

203.1  Overview:  Both  quantitative  and  qualitative  analyses  are 

useful  in  determining  where  fault  detection  resources  should  be  applied. 
The  analyses  identify  improvements  that  must  be  made  if  requirements  are 
to  be  met. 

In  particular,  the  analyses  are  efficient  work  direction  tools  because 
they  can  confirm  system  adequacy  or  identify  the  need  for  design  change, 
provided  they  are  accomplished  in  conjunction  with,  or  reviewed  by,  other 
disciplines . 

203.2  Purpose:  The  purpose  of  Task  203  is  to  develop  a  fault  detection 
model  for  making  numerical  allocations  and  estimates  to  evaluate  system/ 
subsystem/equipment  fault  detection  effectiveness. 


203.3  Task  Description:  A  fault  detection  mathematical  model  based  on 

system/ subsystem/equipment  functions  shall  be  developed  and  maintained. 

As  the  design  evolves,  a  fault  detection  block  diagram  (FIG)  with 
associated  allocations  and  predictions  for  all  elements  in  the  FIG  shall 
be  created.  The  fault  detection  block  diagram  shall  be  keyed  and 
traceable  to  the  functional  block  diagram,  schematics,  drawings,  and 
specifications.  The  model  outputs  shall  be  expressed  in  terms  of  fault 
detection  requirements.  As  changes  occur,  the  model  shall  be  updated  to 
include  hardware  or  software/ firmware  design  changes. 

The  fault  detection  model  shall  be  updated  with  information  resulting 
from  relevant  tests  and  changes  in  item  configuration. 

TASK  204  FAULT  DETECTION  ALLOCATION 

204.1  Overview:  System  fault  detection  requirements  evolve  in  a 

number  of  ways,  from  informed  judgments  to  analyses  based  on  empirical 
data.  The  requirements  are  designed  to  minimize  the  total  cost  of 
developing,  procuring,  and  operating  the  system  over  its  life  cycle.  The 
integrity  of  the  system  is  maintained  by  adequate  top-down  design  that 
ensures  that  the  system  will  meet  specified  requirements.  The  specific 
subsystem  requirements  must  be  refined  before  resources  can  be 
specifically  allocated  for  them. 

204.2  Purpose:  The  purpose  of  Task  204  is  to  ensure  that,  once 

quantitative  system  requirements  have  been  determined,  they  are  properly 
allocated  or  apportioned  to  lower  levels. 
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204.3  Task  Description:  Both  the  mission  and  mission  integrity 
requirements  shall  be  allocated  to  the  level  specified  and  shall  be  used 
to  establish  the  baseline  requirements  for  designers  and  software/ 
firmware  personnel.  Requirements  consistent  with  the  allocations  shall 
be  imposed  on  all  subcontractors  and  suppliers.  The  allocated  values 
shall  be  included  in  appropriate  sections  of  any  procurement 
specifications,  critical  item  specifications,  and  contract  and  item 
specifications  to  subcontractors/suppliers.  All  allocated  fault 
detection  values  established  by  the  contractor  and  included  in  sub¬ 
contract  item  specifications  shall  be  consistent  with  the  mathematical 
model  required  in  Task  203 . 


TASK  205  FA(ji,T  DETECTION  PREDICTION 

205.1  Overview:  Allocations  are  determined  from  the  system  fault 
detection  requirements  to  provide  lower  level  requirements  that  are 
levied  on  the  designers  and  software/f irmware  engineers.  As  design  work 
progresses,  predictions  (based  on  previously  generated  data)  and 
assessments  (based  on  program  test  data)  are  used  to  determine  whether  or 
not  the  allocated  requirement  can  or  will  be  met. 

Predictions  combine  lower  level  fault  detection  data  to  indicate 
equipment  fault  detection  performance  at  successively  higher  levels,  from 
subassemblies  through  subsystem  to  system.  Predictions  falling  short  of 
requirements  at  any  level  signal  the  need  for  management  and  technical 
attention. 

205.2  Purpose:  The  purpose  of  Task  205  is  to  estimate  the  fault 
detection  capability  of  the  system,  subsystem,  equipment,  hardware,  and 
software/f irmware  and  to  determine  whether  or  not  the  fault  detection 
requirements  can  be  achieved  with  the  proposed  design. 

205.3  Task  Description:  Fault  detection  predictions  shall  be  made  for 

the  system,  subsystem,  equipment,  hardware,  and  software/f irmware.  The 
predictions  shall  include  the  probability  of  not  diagnosing  a  fault  and 
the  probability  of  incorrectly  diagnosing  a  fault. 

The  predictions  shall  be  made  by  using  the  associated  fault  detection 
block  diagram  and  fault  detection  coverage  data  and  shall  be  approved  by 
the  Government.  Items  and  equipment  shall  not  be  excluded  from  the 
predictions  for  any  reason. 


TASK  206  FAULT  DETECTION  FAULT  IDENTIFICATION 


206.1  Overview:  Faults  that  are  detected  must  also  be  correctly 
identified.  In  order  to  perform  repair  actions,  much  detail  about  each 
fault  is  required.  The  particular  off-line  tests  using  the  fault 
location  function  which  identify  the  correct  fault  isolation  group  and 
Line  Replacement  Unt  (LRU)  and  possibly  the  failing  LRU  often  require 
more  than  one  fault  location  test  to  be  performed.  For  this  reason,  all 
monitored  test  points  that  provide  fault  information  to  the  central 
PM/FD/FL  function  must  be  correctly  designed.  The  information  from  these 
test  points  must  be  recorded  and  assimilated  into  proper  groupings , which 
identify  the  suitable  fault  location  test  to  be  performed. 

206.2  Purpose:  The  purpose  of  Task  206  is  to  verify  that  proper 

fault  identification,  display,  and  maintenance  action  codes  will 

be  available  to  maintenance  personnel.  Verification  shall  also 
demonstrate  that  the  identity  of  any  faults  detected  will  be  prioritized 
so  that  maintenance  personnel  will  perform  tests  for  the  more  likely 
fault  first.  Verification  shall  show  that  the  correct  information 
specified  in  the  PIDS  for  each  detected  fault  is  correctly  provided  to 
and  displayed  on  the  maintenance  panel. 

206.3  Task  Description:  A  fault  detection  fault  identification 
plan  shall  be  developed  and  include,  but  not  be  limited  to 

1.  A  description  of  how  fault  identification  is  handled  by 
the  system. 

2.  A  description  of  how  the  fault  identification  design 
meets  PIDS  requirements 

3.  A  te6t  plan  and  procedure  for  proper  fault 
identification. 

4.  A  worst-case  series  of  tests  to  show  that  the  most 
likely  fault  is  displayed  first. 

5.  Test  cases  intended  to  be  ambiguous  with  respect  to 
which  fault  initiated  the  problem. 

6.  Stress  tests  for  proper  fault  identification  under 
actual  operating  conditions. 
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TASK  207  FAULT  DETECTION  FUNCTION  CERTIFICATION 


207.1  Overview:  It  is  vital  that  designs  be  tested  not  only  to  see  if 
the  designs  themselves  are  functional  and  fault  free  but  also  that  the 
designs  meet  not  only  the  'letter  of  specification'  it  also  meet  the 
actual  intent  of  the  specification.  Verification  of  asign  to 
specification  should  be  verified  at  all  levels  of  de  alopment  and  when  it 
appears  to  have  been  completed,  retesting  and  reveri  ication  should 
occur,  starting  at  the  original  design  teams,  to  con  ractor  quality 
assurance  personnel,  to  independent  test  teams,  and  Lnally  by  the 
Government . 

207.2  Purpose:  The  purpose  of  this  task  is  to  veri  /  and  demonstrate 
that  the  design  for  the  fault  detection  subfunction  aets  not  only  the 
'letter  of  the  specification'  but  also  meets  the  int  it  of  the 
specification. 

207.3  Task  Description:  Task  207  fault  detection  inction 

certification  is  a  series  of  qualifying  tests  to  det  -mine  adherence  to 
the  PIDS  requirements.  These  tests  shall  be  designe  to  answer,  as  a 
minimum,  the  following  questionsas  my  be  required  by  :he  specif ications : 

1.  Did  the  fault  detection  function  detect  th  fault? 

2.  Did  fault  detection  provide  effective  faul  isolation 
information  for  corrective  maintenance  act  ms? 

3.  Did  fault  detection  provide  information  fc  further 
tests  which  could  confirm  the  problem? 

4.  Did  the  fault  detection  function  provide  :  ormation 
regarding  the  impact  of  the  fault  to  the  s  item? 

5.  What  was  the  latency  time  between  the  occu  *ence  of  the 
fault  and  the  final  indication  on  the  pane 

6.  Was  there  any  ambiguity  surrounding  the  fa  -t  or  the 
correction? 

7.  What  are  the  total  number  of  undetected  fa  -ts  in  any 
given  period?  Why  were  they  not  detected? 

8.  What  is  the  latency  time  from  software/har  -/are  failure 
to  automatic  rebooting? 

9.  What  is  the  latency  time  to  detect  a  probi  n  in  the 
computer  firmware/hardware  that  is  not  cor  actable  by 
automatic  rebooting? 


-21- 


TR  8315 


TASK  208  FAULT  DETECTION  INDEPENDENT  VERIFICATION  AND  VALIDATION 

208.1  Overview:  Independent  fault  detection  verification  and  validation 
performed  by  a  scientific  team  not  involved  in  the  design,  development, 
and  tests  ensures  that  the  fault  detection  design  meets  the  PIDS 
requirements.  The  IV&V  team  will  ensure  that  the  fault  detection 
subprogram  will  not  fail  and  will  perform  up  to  its  intended  capacity. 

208.2  Purpose:  The  purpose  of  Task  208  is  to  independently  determine 

that  the  PIDS  and  SOW  requirements  have  been  met. 

208.3  Task  Description:  Procedures  shall  be  independently  established, 

maintained,  and  implemented,  to  be  performed  by  test  and  analysis,  to 
verify  and  validate  the  ability  of  the  fault  detection  subsystem  to  meet 
all  of  the  requirements  of  the  PIDS  and  SOW.  The  functional  testing  of 
the  design  shall  employ  methodologies  of  great  stress  and  strain  to  the 
hardware  and  firmware/software.  The  fault  detection  subsystem  shall  be 
tested  under  worst-case  actual  operational  conditions.  The  documentation 
produced  by  the  independent  IV&V  team  shall  include,  but  not  be  limited 
to 

1.  The  test  plan  for  the  tests  which  will  be  conducted, 
including  the  operational  conditions  under  which  they 
will  be  performed. 

2.  The  actual  test  procedures  with  dates,  test  engineer, 
location,  and  all  other  pertinent  information. 

3.  Identification,  description,  listings,  and  source  code 
for  IV&V  test  programs. 

4.  Complete  test  reports,  results,  deficiencies,  problems, 
and  observations. 

TASK  209  FAULT  DETECTION  CONFIGURATION  MANAGEMENT 

209.1  Overview:  Separate  plans  and  procedures  shall  be  prepared  and 
adopted  for  the  collecting,  cataloging,  describing,  of  all  designs, 
changes,  implementations,  problems,  programs,  test  procedures,  test 
results,  test  findings,  conclusions,  and  observations  for  the  fault 
detection  program,  subprogram,  elements,  hardware  firmware/software. 

209.2  Purpose:  The  purpose  of  this  task  is  to  verify  and  demonstrate 
the  configuration  management  specifications,  detail,  quantity,  quality, 
and  media  are  sufficient  meet  the  remi^oment.s  for  the  program. 
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209.3  Task  Description:  The  contractor  shall  use  a  configuration 

management  program  for  the  fault  detection  system,  subsystem,  program 
elements,  hardware,  software/firmware,  ECPs,  PDRs,  listings  of  PIDs 
requirements  (as  interpreted  by  the  contractor),  and  any/all  other 
documentation  pertinent  to  the  fault  detection  system.  Data  shall  also 
include  but  not  be  limited  to 

1.  Requirements  as  provided  to  subcontractors. 

2.  Subcontractors  response  and  interpretation  of  requirements, 

3.  Test  procedures  by  contractor,  and  subcontractors. 

4.  Any  qualification  tests,  results,  conclusions,  and/or 
observations . 


5.  Changes  as  provided  by  the  program  office,  as  initiated  by 
the  contractor,  as  required  by  the  results  from  new  data, 
as  required  for  any  other  purposes. 


6.  All  data  item  requirements. 

7.  All  data  necessary  for  life  cycle  support,  test  certification. 

8.  Design  drawings,  source  code,  program  language(s),  and  other 
documentation  to  provide  the  capability  for  independent 
certification,  duplication  of  the  system,  subsystem,  elements, 
firmware/software,  and  hardware. 


j  TASK  210  FAULT  DETECTION  FAULT  IMPACT 

210.1  Overview:  Not  all  faults  have  the  same  effect  on  system 
integrity,  system  effectiveness,  or  system  operational  availability. 

Some  faults  mask  other  faults  that  may  have  more  of  an  impact  on  system 
integrity.  Similarly,  certain  portions  of  systems  have  redundancies, 
either  natural  or  planned.  In  the  case  of  faults  in  redundant  portions, 
it  may  be  possible  to  schedule  maintenance  for  some  planned  time.  The 
faults,  then,  are  not  critical  at  that  time  to  system  integrity  or 
operations,  provided  they  are  recorded  and  repaired  at  the  next  repair 
cycle  time.  When  a  multitude  of  faults  occurs,  there  are  often  one  or 
two  major  faults  which  have  had  a  ripple  effect  and  cause  other  faults  to 
occur.  The  ripple  effect  is  potentially  dangerous  because  the  impact  on 
system  operation  will  not  be  easily  determined  and  the  parent  fault(s)  of 
the  problem  may  not  be  identified.  By  assigning  levels  of  impact  to  each 
fault,  there  is  a  better  probability  of  correctly  assessing  the  fault 
impact,  determining  system  impact,  and  looking  for  the  most  damaginr 
fault  first. 

i 

i 
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In  effect,  giving  a  level  of  impact  to  each  fault  allows  for  more 
correct  diagnosis  of  the  actual  cause  of  failures.  For  example,  if  a 
power  supply  were  to  be  at  fault,  most  of  the  units  that  had  test  points 
for  the  fault  detection  subsystem  would  give  indication  of  failure.  For 
this  reason,  given  the  multitude  of  possible  faults  occurring  or  seeming 
to  occur  all  at  once,  it  is  necessary  to  determine  the  impact  of  every 
test  point  utilized  for  the  fault  detection  subsystem.  The  standard 
procedure  is  to  give  each  fault  an  impact  level  (sometimes  called  a 
priority  level). 

210.2  Purpose:  The  purpose  of  Task  207  is  to  test  the  ability  of  the 

fault  detection  subsystem  to  correctly  identify  faults  it  has  detected 
and  to  correctly  determine  the  impact  of  those  faults  with  respect  to  the 
integrity  and  effectiveness  of  the  major  system.  Additionally,  the  task 
is  to  demonstrate  that  faults  do  not  mask  each  other  when  they  occur  at 
the  same  time. 

210.3  Task  Description:  A  fault  detection  plan  for  fault  impact  shall 
be  developed  and  shall  include  but  not  be  limited  to 

1.  A  description  of  how  the  fault  detection  design  meets 
the  PIDS  requirements  with  respect  to  fault  impact. 

2.  A  description  of  how  fault  impact  is  handled  by  the 
system. 

3.  A  description  of  the  assignment  of  priority  to  faults 
with  respect  to  fault  impact. 

4.  A  test  plan  and  procedure  for  testing  fault  detection 
and  fault  impact. 

5.  A  worst-case  series  of  tests  and  their  evaluations. 

6.  Test  cases  intended  to  be  ambiguouf  ith  respect  to 
which  fault  initiated  the  problem. 

7.  Stress  test  cases  under  actual  operating  conditions. 

8.  A  listing  of  test  panel  indications  for  all  above  tests. 

Documentation  for  all  fault  impact  test  results  shall  be  included 
in  this  task. 
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TASK  211  FAULT  DETECTION  FUNCTION  TRANSIENT  SMOOTHING 

211.1  Overview:  Electronic  systems,  especially  those  that  have 

long  distances  between  units,  are  susceptible  to  all  kinds  of 
interference,  including  DC  offsets,  ground  loops,  EMI,  and  noise 
bursts  and  pulses  caused  by  other  electronic  devices.  The  devices 
themselves  may  also  cause  transients  when  certain  combinations  of 
operations  are  performed.  Therefore,  a  simple  pass/fail  test  at  any  test 
point  may  show  indication  of  a  fault  when,  in  fact,  there  is  none. 
Similarly,  a  fault  finding  may  be  lost  or  erroneously  modified  during 
transmission  from  one  system  component  to  another.  Transient  smoothing 
is,  therefore,  required  to  reduce  the  number  of  false  fault  indications. 
It  is  also  imperative  that  certain  test  points  which  are  critical  to 
system  integrity  have  their  responses  quickly  read.  All  test  points 
should  be  able  to  report  within  given  latency  times  even  if  anomalies 
exist  somewhere  in  the  subsystem. 

211.2  Purpose:  The  purpose  of  Task  211  is  to  ensure  the  ability 

of  the  fault  detection  subsystem  to 

1.  Report  all  faults  within  the  specified  latency  time, 
regardless  of  anomalies,  either  at  the  test  point  or 
during  transmission  from  one  point  to  another. 

2.  Report  the  condition  of  any  test  point  that  has  become 
inoperative  or  incommunicative. 

3.  Not  report  non-recurring  faults,  glitches,  or 
transients . 

211.3  Task  Description:  A  Fault  Detection  Transient  Smoothing 

Plan  for  design,  test,  certification,  and  verification  shall  be 
developed  and  implemented.  The  plan  shall  include  but  not  be 
limited  to 

1.  A  description  of  how  each  fault  is  handled  to  avoid 
false  alarms. 

2.  A  description  of  verification/validation  test  plans  for 
transient  smoothing. 

3.  A  description  of  verification  of  tests  to  be  performed 
under  worst-case  actual  operating  conditions  or 
equivalent . 

4.  A  description  of  the  verification  test  that  ensures  the 
reporting  of  faults  within  the  time  specified  in  the 
PIDS. 


A  report  on  the  implementation  of  this  plan,  including  test 
findings,  shall  be  included  in  all  design  reviews. 
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FAULT  LOCALIZATION  TASKS 
TASK  301  FAULT  LOCALIZATION  PROGRAM  PLAN 

301.1  Overview:  The  Fault  Localization  Program  Plan  shall  be 
designed  as  a  basic  tool  to  assist  the  contractor  in  implementing 

a  fault  localization  development  program.  The  Government  will  also 
use  the  plan  to  (1)  evaluate  the  contractor's  approach  to,  and  his 
execution  of,  fault  localization  tasks  (2)  evaluate  the  adequacy 
of  his  procedures  for  planning,  implementing,  and  controlling  the 
fault  localization  tasks  and  (3)  evaluate  the  ability  of  his 
organizational  structure  to  focus  on  fault  location  activities/ 
problems . 

301.2  Purpose:  The  purpose  of  Task  301  is  to  develop  a  Fault  Local¬ 
ization  Program  Plan  that  identifies  and  integrates  all  program 
tasks  necessary  to  accomplish  fault  localization  requirements  of  the 
Prime  Item  Development  Specification  (PIDS)  and  the  Statement  of 
Work  (SOW). 

301.3  Task  Description:  A  fault  localization  Program  Plan  shall  be 
prepared  to  provide,  as  a  minimum,  the  following: 

1.  A  description  of  how  the  fault  localization  program  will  be 
conducted  to  meet  the  requirements  of  the  PIDS  and  the 
SOW. 

2.  A  description  of  how  fault  localization  design  interfaces 
with  total  system  design. 

3.  A  detailed  description  of  how  each  specific  fault 
localization  requirement  will  be  performed  or  complied  with. 

4.  The  procedures  to  evaluate  the  status  and  control  of 
each  task,  and  identification  of  the  organizational  unit 
with  the  authority  and  responsibility  for  executing  each 
task. 

5.  A  schedule  with  estimated  start  and  completion  points 
for  each  fault  localization  program  activity  or  task. 

6.  The  identification  of  known  fault  localization  problems  to 
be  solved,  an  assessment  of  the  impact  of  these  problems 
on  meeting  specified  requirements,  and  the  proposed 
solutions  or  proposed  plan  to  solve  them. 
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7.  The  procedure  or  methods  for  recording  the  status  of 
actions  to  resolve  problems. 

8.  The  designation  of  fault  localization  milestones,  including 
design  review  (PDR,  CDR,  IPR)  and  test. 

9.  The  method  by  which  the  fault  localization  requirements  are 
disseminated  to  designers  and  associated  personnel  and 

how  design  interfaces  are  accomplished. 

10.  Identification  of  key  personnel  for  managing  the  fault 
localization  program  and  the  level  of  authority  for  problem 
resolution. 

11.  Description  of  the  management  structure,  including 
interrelationship  between  line,  service,  staff,  and 
policy  organizations. 

12.  The  fault  localization  design  review  checklist  that  will  be 
used  to  ensure  that  the  design  meets  requirements. 


When  approved  by  the  Government,  the  fault  localization  Program  Plan 
shall  become  a  basis  for  evaluation  of  contractual  compliance. 

TASK  302  FAULT  LOCALIZATION  PROGRAM  DESIGN  REVIEWS 

302.1  Overview:  Periodic  design  reviews  should  be  held  to  establish 
whether  or  not  the  projected  design  will  meet  the  requirements  of  the 
specifications.  At  the  onset,  reviews  should  be  held  more  frequently 
to  ensure  that  the  contractor  does  not  proceed  with  unsuitable 
designs.  The  reviews  are  also  to  confirm  that  the  contractor  is  not 
only  meeting  the  'wording'  of  the  specification,  but  also  the  intent 
of  the  specification.  Design  reviews  may  be  held  at  any  time,  and  it 
is  not  necessary  that  they  be  separate  from  other  reviews,  providing 
that  they  are  given  proper  emphasis  as  would  be  required  to  ensure 
that  the  contractor  is  performing  and  adhering  to  Government 
standards  and  requirements  and  also  to  other  sections  of  this  entire 
specification. 

302.2  Purpose:  The  purpose  of  Task  302  is  to  establish  a  requirement 
for  the  contractor  to  conduct  formal  and  informal  fault  localization 
program  design  reviews. 


/*yximntvnyi» 


TR  8315 


302.2  Task  Description:  Fault  localization  formal  design  reviews 
shall  be  conducted  in  accordance  with  the  requirements  of  MIL-STD- 
1521B  on  a  schedule  approved  by  the  Government.  Informal  fault 
localization  in-process  reviews  shall  be  conducted  at  least  quarterly 
until  formal  CDR  on  a  schedule  mutually  agreed  upon  by  the  government 
and  the  contractor.  The  contractor-proposed  formal  and  informal 
design  review  schedule  shall  be  provided  as  part  of  the  fault 
localization  Program  Plan. 

In  addition  to  the  formal  design  review  requirements  of  MIL-STD- 
1521B,  the  following  formal  and  informal  design  reviews  shall  include 
the  fault  localization  requirements  indicated  below: 

1.  Preliminary  Design  Review  (PDR): 

a.  Updated  fault  localization  program  status,  including 

1)  fault  localization  modeling; 

2)  fault  localization  allocation; 

3)  fault  localization  predictions; 

4)  fault  localization  compliance  with  specifications; 

5)  design  guideline  criteria. 

b.  Problems  affecting  fault  localization. 

c.  Fault  localization  critical  items. 

2.  Critical  Design  Review  (CDR): 

a.  Fault  localization  compliance  with  specifications. 

b.  Fault  localization  predictions  and  analyses. 

c.  Fault  localization  critical  items. 

d.  Problems  affecting  fault  localization. 

e.  Identification  of  circuits  where  the  design  requires 
high  reliability  components  and  the  software/ 
firmware  employs  an  extra-large  number  of  lines  of 
code. 

3.  In-Process  Fault  Localization  Reviews  (IPR): 

a.  Discussion  of  those  fault  localization  items  previously 
listed  under  Sections  a  and  b. 

b.  Results  of  fault  localization  test  analyses. 

c.  Test  schedule:  start  and  completion  dates. 
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d.  Fault  localization  parts,  design,  reliability,  and 
schedule  problems . 

e.  Status  of  assigned  action  items. 

f.  Contractor's  assessment  of  fault  localization  design 
effectiveness . 

g.  Other  topics  and  issues  on  the  agenda  agreed  to  by 
the  contractor  and  the  Government . 

h.  Results  of  applicable  fault  localization  growth  testing. 


4.  Test  Readiness  Review: 


^  , 

Fault  Localization  analyses 
prediction . 

status  and  primary 

b. 

Test  schedule. 

c. 

Test  profile. 

d. 

Test  plan  including  failure 

definition. 

e. 

Test  report. 

5.  Production  Readiness  Review:  Results  of  applicable  fault 
localization  growth  testing. 

TASK  303  FAULT  LOCALIZATION  MODELING 

303.1  Overview:  Both  quantitative  and  qualitative  analyses  are 
useful  in  determining  where  fault  localization  resources  should  be 
applied.  The  analyses  identify  improvements  that  must  be  made  if 
requirements  are  to  be  met . 

In  particular,  the  analyses  are  efficient  work  direction  tools 
because  they  can  confirm  system  adequacy  or  identify  the  need  for 
design  change,  provided  they  are  accomplished  in  conjunction  with,  or 
reviewed  by,  other  disciplines. 

303.2  Purpose:  The  purpose  of  Task  303  is  to  develop  a  fault 
localization  model  for  making  numerical  allocations  and  estimates  to 
evaluate  system/subsystem/equipment  fault  localization  effectiveness. 
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303.3  Task  Description:  A  fault  localization  mathematical  model 
based  on  system/ subsystem/equipment  functions  shall  be  developed  and 
maintained.  As  the  design  evolves,  a  fault  localization  block  diagram 
(fault  isolation  groupings)  with  associated  allocations  and  predict¬ 
ions  for  all  elements  in  the  FIG  shall  be  created.  The  fault 
localization  block  diagram  shall  be  keyed  and  traceable  to  the 
functional  block  diagram,  schematics,  drawings,  and  specifications. 
The  model  outputs  shall  be  expressed  in  terms  of  fault  localization 
requirements.  As  changes  occur,  the  model  shall  be  updated  to 
include  hardware  or  software/f irmware  design  changes. 

The  fault  localization  model  shall  be  updated  with  information 
resulting  from  relevant  tests  and  changes  in  item  configuration. 

TASK  304  FAULT  LOCALIZATION  ALLOCATION 

304.1  Overview:  System  fault  localization  requirements  evolve  in  a 
number  of  ways,  from  informed  judgments  to  analysis  based  on 
empirical  data.  The  requirements  are  designed  to  minimize  the  total 
cost  of  developing,  procuring,  and  operating  the  system  over  its  life 
cycle.  The  integrity  of  the  system  is  maintained  by  adequate  top- 
down  design  that  ensures  the  ability  of  the  system  to  meet  specified 
requirements.  The  specific  subsystem  requirements  must  be  refined 
before  resources  can  be  specifically  allocated  for  them. 

304.2  Purpose:  The  purpose  of  Task  304  is  to  ensure  that,  once 
quantitative  system  requirements  have  been  determined,  they  are 
allocated  or  apportioned  to  lower  levels . 

304.3  Task  Description:  Both  the  mission  and  mission  integrity 
requirements  shall  be  allocated  to  the  level  specified  and  shall 
be  used  to  establish  the  baseline  requirements  for  designers  and 
software/firmware  personnel.  Requirements  consistent  with  the 
allocations  shall  be  imposed  on  all  subcontractors  and  suppliers. 

The  allocated  values  shall  be  included  in  appropriate  sections  of 
any  procurement  specifications,  critical  item  specifications,  and 
contract  end  item  specifications  to  subcontractors/suppliers. 

The  allocated  values  shall  be  included  in  appropriate  sections  of  any 
procurement  specifications,  critical  item  specifications,  and 
contract  end  item  specifications  to  subcontractors/suppliers.  All 
allocated  fault  localization  values  established  by  the  contractor  and 
included  in  subcontract  item  specifications  shall  be  consistent  with 
the  mathematical  model  required  in  Task  303. 
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TASK  305  FAULT  LOCALIZATION  PREDICTION 


305.1  Overview:  Allocations  are  determined  from  the  system  fault 
localization  requirements  to  provide  lower  level  requirements  that 
are  levied  on  the  designers  and  software/ firmware  engineers.  As 
design  work  progresses,  predictions  (based  on  previously  generated 
data)  and  assessments  (based  on  program  test  data)  are  used  to 
determine  whether  or  not  the  allocated  requirement  can  or  will  be 
met . 

Predictions  combine  lower  level  fault  localization  data  to  indicate 
equipment  fault  localization  performance  at  successively  higher 
levels,  from  subassemblies  through  subsystem  to  system.  Predictions 
falling  short  of  requirements  at  any  level  signal  the  need  for 
management  and  technical  attention. 

305.2  Purpose:  The  purpose  of  Task  305  is  to  estimate  the  fault 
location  capability  of  the  system,  subsystem,  equipment,  hardware, 
and  software/firmware  and  to  determine  whether  or  not  the  fault 
localization  requirements  can  be  achieved  with  the  proposed  design. 

305.3  Task  Description:  Fault  localization  predictions  shall  be 

made  for  the  system,  subsystem,  equipment,  hardware,  and  software/ 
firmware.  The  predictions  shall  include  (1)  the  probability  of  not 
localizing  the  fault;  (2)  the  probability  of  localizing  a  fault  to 
the  incorrect  fault  isolation  group;  and  (3)  the  probability  of 
localizing  a  fault  to  within  the  correct  fault  isolation  group. 

The  predictions  shall  be  made  by  using  the  associated  fault 
localization  block  diagram  and  fault  localization  coverage  data  and 
shall  be  approved  by  the  Government.  Items  and  equipment  shall  not 
be  excluded  from  the  prediction  for  any  reason. 

TASK  306  FAULT  LOCALIZATION  FAULT  IDENTIFICATION 

306.1  Overview:  Many  faults  cause  domino  effects  where  the 
occurrence  of  one  fault  causes  additional  other  fault  indications. 

In  order  to  provide  effective  repair,  in  minimum  time,  and  also  to 
evaluate  the  impact  on  the  system  caused  by  the  root  fault,  it  is 
necessary  that  the  root  fault  be  determined  and  found.  The  design  of 
the  fault  localization  subsystem  must  be  of  sufficient  complexity  to 
isolate  the  root  fault  despite  the  occurrence  of  multiple  faults  and 
other  ambiguities. 

306.2  Purpose:  The  purpose  of  Task  306  is  to  test  the  ability 

of  the  fault  localization  subsystem  to  detect  and  correctly  identify 
faults . 
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306.3  Task  Description:  A  fault  localization  plan  for  fault 

identification  shall  be  developed  that  includes  but  is  not  limited  to 

1.  A  description  of  how  fault  identification  is  handled  by 
the  system. 

2.  A  description  of  how  the  fault  localization  design  meets 
the  PIDS  requirements. 

3.  A  test  plan  and  procedure  for  testing  fault 
identification . 

4.  A  worst-case  series  of  tests  and  their  evaluations. 

5.  Test  cases  intended  to  be  ambiguous  with  respect  to 
which  fault  initiated  the  problem. 

6.  Stress  test  cases  under  actual  operating  conditions. 

7.  A  listing  of  the  test  panel  indications  for  all  above 
tests . 

Documentation  of  all  fault  localization  test  results  shall  be 
included  in  this  task. 

TASK  307  FAULT  LOCALIZATION  FUNCTION  CERTIFICATION 

307.1  Overview:  It  is  vital  that  designs  be  tested,  not  only  to  see 
if  the  designs  themselves  are  functional  and  fault  free,  but  also 
that  the  designs  meet  not  only  the  'letter  of  the  specification'  but 
also  meet  the  actual  intent  of  the  specification.  Verification  of 
design  to  specification  should  be  verified  at  all  levels  of 
development  and  when  it  appears  to  have  been  completed,  retesting  and 
verification  should  occur,  starting  at  the  original  design  team,  to 
contractor  quality  assurance  personnel,  to  independent  test  teams, 
and  finally  by  the  Government. 

307.2  Task  Description:  Task  307  Fault  localization  function 
certification  is  a  series  of  qualifying  tests  to  determine  adherence 
to  the  PIDS  requirements.  These  tests  shall  be  designed  to  answer, 
as  a  minimum,  the  following  questions: 

1.  Did  the  fault  localization  function  detect  the  fault? 

2.  Did  fault  localization  provide  effective  fault  isolation 
information  for  corrective  maintenance  actions? 

3.  Did  fault  localization  provide  information  for  further 
tests  that  could  confirm  the  problem? 

4.  Was  there  any  ambiguity  surrounding  the  fault  or  the 
correction? 

5.  Were  there  any  unlocalized  faults?  Why  were  they  not 
localized? 
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TASK  308  FAULT  LOCALIZATION  INDEPENDENT  VERIFICATION  &  VALIDATION 

308.1  Overview:  Independent  fault  localization  verification  and 
validation  performed  by  a  scientific  team  not  involved  in  the  design, 
development,  and  tests  ensures  that  the  fault  localization  design 
meets  the  PIDS  requirements.  The  IV&V  team  will  ensure  that  the 
fault  localization  subprogram  will  not  fail  and  will  perform  up  to 
its  intended  capacity. 

308.2  Purpose:  The  purpose  of  Task  308  is  to  independently 
determine  that  the  PIDS  and  the  SOW  requirements  have  been  met. 

308.3  Task  Description:  Procedures  shall  be  independently 

established,  maintained,  and  implemented,  to  be  performed  by  test 
and  analysis,  to  verify  and  validate  the  ability  of  the  fault 
localization  subsystem  to  meet  all  requirements  of  the  PIDS  and  SOW. 

The  fault  localization  subsystem  shall  be  tested  under  worst-case 
actual  operational  conditions.  The  documentation  produced  by  the 
IV&V  team  shall  include  but  not  be  limited  to 

1.  The  test  plan  for  the  tests  which  will  be  conducted, 
including  the  operational  conditions  under  which  they 
will  be  performed. 

2.  The  actual  test  procedures  with  dates,  test  engineer, 
location,  and  all  other  pertinent  information. 

3.  Identification,  description,  listings,  and  source  code 
for  IV&V  test  programs  are  used. 

d.  Complete  test  reports,  results,  deficiencies,  problems, 
and  observations. 


TR  83i5 

TASK  309  FAULT  LOCALIZATION  CONFIGURATION  MANAGEMENT  OVERVIEW: 

309.1  Overview:  Separate  plans  and  procedures  shall  be  prepared  and 
adopted  for  the  collecting,  cataloging,  and  describing  of  all 
designs,  changes,  implementations,  problems,  programs,  test 
procedures,  test  results,  test  findings,  conclusions,  observations, 
for  the  fault  localization  program,  subprogram,  elements,  hardware 

f irmware/sof tware . 

309.2  Purpose:  The  purpose  of  this  task  is  to  verify  and 
demonstrate  the  configuration  management  specifications,  detail, 
quantity,  quality,  and  media  are  sufficient  to  meet  the  requirements 
for  the  program. 

309.3  Task  Description:  The  contractor  shall  use  a  configuration 

management  program  for  the  fault  localization  system,  subsystem, 
program  elements,  hardware,  software/firmware,  ECPs,  PDRs,  listings 
of  PIDs  requirements  ( as  interpreted  by  the  contractor),  any/all 
other  documentation  pertinent  to  the  fault  localization  system.  Data 
shall  also  include,  but  not  be  limited  to: 

1.  Requirements  as  provided  to  subcontractors. 

2.  Subcontractors  response  and  interpretation  of  requirements. 

3.  Contractor  and  subcontractors  test  procedures. 

4.  Any  qualification  tests,  results,  conclusions,  and/or 
observations . 

5.  Any  changes  as  provided  by  the  program  office,  as  initiated 
by  the  contractor,  as  required  by  the  results  from  new  data, 
as  required  for  any  other  purposes. 

6.  All  data  item  requirements. 

7.  All  data  necessary  for  life  cycle  support,  test,  and 
certification. 

8.  Design  drawings,  source  code,  program  language(s),  and  other 
documentation  used  to  provide  the  capability  for  independent 
certification,  duplication  of  the  system,  subsystem, 
elements,  f irmware/sof tware,  and  hardware. 
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CONCLUS IONS/SUMMARY : 


The  continuing  increase  in  complexity  of  military  systems  has  imposed 
additional  maintenance  and  logistic  burdens  on  our  operating  forces.  As  these 
organizations  experience  a  reduction  of  both  manning  and  skill  levels  the 
requirements  for  quick  equipment  malfunction  repair  has  risen.  Experience  has 
indicated  that  when  systems  become  fully  operational,  a  number  of  problems  are 
likely  to  occur  and  that  these  problems  must  be  dealt  with  in  an  orderly, 
precise  and  cost  effective  manner.  As  a  system  matures,  problems  still  exist 
and  all  but  the  simplest  will  pose  insurmountable  difficulties  to  the  test  and 
repair  technician(s) .  Also,  because  major  turnovers  in  experienced  personnel 
is  a  fact  that  cannot  be  dismissed,  automated  performance  monitoring,  fault 
detection  and  fault  localization  for  sustaining  day-to-day  support  of  a  system 
must  be  accomplished  by  the  user  organization,  namely  our  operating  fleet.  The 
report  presented  herein,  therefore,  presents  a  method  of  developing  performance 
and  maintenance  aid  design  techniques  that  enables  the  system  to  localize 
faults  to  a  manageable  number  of  units.  The  technique  shown  provides  a  record 
of  the  design  elements  requisite  to  best  design  practices  and  provides  a 
systematic  approach  to  the  PM/FD/FL  process  not  previously  provided  in  contract 
or  SOW  requirements.  Incorporation  of  this  document  and/or  portions  thereof 
into  system  SOW  documentation  will  allow  relevant  subject  areas  to  be  addressed 
and  judgements  of  conformity  to  requirements  can  be  more  readily  made  by  the 
reviewing  agency. 

The  specification  as  described  herein  has  been  successfully  applied  to  a 
Navy  sponsored  program.  Elements,  as  developed,  were  collected  and  combined 
resulting  in  the  subject  document  for  the  purpose  of  future  application  in 
programs  requiring  PM/FD/FL  design/development. 
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STATEMENT  OF  WORK  SAMPLES 


PERFORMANCE  MONITORING,  FAULT  DETECTION, 
FAULT  LOCALIZATION  REQUIREMENTS 


1.0  Performance  Monitoring,  Fault  Detection,  Fault  Localization 
Program. 

The  contractor  shall  develop  and  implement  a  PM/FD/FL  program  in 
accordance  with  the  statement  of  work  (SOW)  and  the  tailored 
requirements  of  the  applicable  PIDS,  and  where  applicable  pertinent 
sections  of  MIL-STD-785B,  MIL-STD-470A,  MIL-STD-2167A. 


1.1  PERFORMANCE  MONITORING 

a.  Task  101  Performance  Monitoring  Program  Plan. 

b.  Task  102  Performance  Monitoring  Program  Design  Reviews. 

c.  Task  103  Performance  Monitoring  Modeling. 

d.  Task  104  Performance  Monitoring  Allocation. 

e.  Task  105  Performance  Monitoring  Prediction. 

f.  Task  106  Performance  Monitoring  Fault  Tree. 

g.  Task  107  Performance  Monitoring  Function  Certification. 

h.  Task  108  Performance  Monitoring  Independent  1V&V. 

i.  Task  109  Performance  Monitoring  Configuration  Management. 

j.  Task  110  Performance  Monitoring  Fault  Impact. 
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FAULT  DETECTION 

a. 

Task 

201 

Fault 

Detection 

Program  Plan. 

b. 

Task 

202 

Fault 

Detection 

Program  Design  Reviews. 

c. 

Task 

203 

Fault 

Detection 

Modeling . 

d. 

Task 

204 

Fault 

Detection 

Allocation. 

e. 

Task 

205 

Fault 

Detection 

Prediction. 

f . 

Task 

206 

Fault 

Detection 

Fault  Identification. 

g. 

Task 

207 

Fault 

Detection 

Function  Certification. 

h. 

Task 

208 

Fault 

Detection 

Independent  1V&V. 

i. 

Task 

209 

Fault 

Detection 

Configuration  Management. 

3- 

Task 

210 

Fault 

Detection 

Fault  Impact. 

k. 

Task 

211 

Fault 

Detection 

Function  Transient  Smoothing 

1.3  FAULT  LOCALIZATION 


a. 

Task 

301 

Fault 

Localization 

Program  Plan. 

b. 

Task 

302 

Fault 

Localization 

Design  Reviews. 

c . 

Task 

303 

Fault 

Localization 

Modeling . 

d. 

Task 

304 

Fault 

Localization 

Allocation . 

e. 

Task 

305 

Fault 

Localization 

Prediction . 

f . 

Task 

306 

Fault 

Localization 

Fault  Identification. 

g- 

Task 

307 

Fault 

Localization 

Function  Certif ication 

h . 

Task 

308 

Fault 

Localization 

Independent  1V&V. 

i . 

Task 

309 

Fault 

Localization 

Configuration  Management 

1 . 0  SCOPE 


1.1  PURPOSE 

This  appendix  to  the  SOW  specifies  the  requirements  to  be 
applied  during  the  development  and  certification  of  the  PM/FD/FL 
subsystem. 

1.2  ADHERENCE  TO  MIL  STANDARDS 

The  software  and  firmware  portions  of  PM/FD/FL  shall  be 
developed,  verified,  validated,  and  certified  as  described  in 
the  SOW.  For  purposes  of  legality,  configuration,  and 
where  else  applicable  the  software/firmware  developed  for  PM/FD/FL 
shall  be  considered  to  be  'tactical'  software/firmware. 
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The  hardware  portions  of  PM/FD/FL  shall  adhere  to  Appendix  B  of 
the  SOW  and  conform  and  be  certified  to  all  requirements  as  stated. 

1.2  SPECIAL  REQUIREMENTS  FOR  PM/FD/FL 

The  system  shall  utilize  the  PM/FD/FL  subsystem  for  the  main¬ 
tenance  demonstration.  The  PM/FD/FL  shall  be  certified  and  accepted 
prior  to  the  maintenance  demo. 

The  software/firmware  portions  of  the  system  shall  conform  to 
reliability  growth  methodology  in  that  progressive  builds,  threads, 
strings  shall  indicate  the  ability  of  the  PM/FD/FL  subsystem  to  meet 
PIDS  requirements  throughout  the  test,  and  certification  programs  and 
also  conform  to  life  cycle  requirements  as  stated  in  the  PIDS,  type- 
A  specifications,  SOW  and  all  appendices,  and  the  contract. 

1.3  PM/FD/FL  DEFAULT  CONDITIONS 

The  PM/FD/FL  subsystem  shall  conform  to  PIDS  requirements  for 
the  automatic  rebooting  of  software,  programs,  parameters, 
executives,  and  all  other  operational  functions.  The  PM/FD/FL 
demonstration  during  both  the  software/ firmware  demo  and  the  PM/FD/FL 
demo  shall  provide  adequate  testing,  certification  and  validation  to 
indicate  that  the  automatic  reboot  design  meets  PIDs  requirements. 

The  default  for  power  failure,  or  failure  of  the  PM/FD/FL  to 
perform  as  required  shall  cause  a  system  failure  indication  or  alarm 
on  all  panels  of  the  system.  Development  of  the  PM/FD/FL  subsystem 
shall  require  indication  that  power  failure,  either  accidental  or 
willful,  shall  cause  an  immediate  failure  indication  condition. 
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Enhanced  PM/FD 


Fault  Detection 


Fault  Impact 


Fault  Isolation  Group 


Imminent  Failure 


Latency  Time 


Monitored  Fault 


Non-invasive 


Performance  Monitoring 


PM/FD/FL  Methodologies 


APPENDIX  B 
GLOSSARY  OF  TERMS 


Additional  software  and/or  hardware 
to  improve  the  probability  of  detecting 
faults. 

That  specific  subfunction  that  detects 
faults  {Separate  or  combined  with  the 
performance  monitoring  subfunction) .  Usually 
employs  white-box  methodologies  where  specific 
test  points  are  selected  that  should  give 
indication  as  to  the  condition  of  the 
electronic  unit  under  test.  Some  fault 
detection  test  points  are  naturally  occurring 
in  design,  others  are  specifically  planned. 
Problems  detected  by  the  fault  detection  sub¬ 
function  may  or  may  not  be  the  basis  of 
determining  system  failure,  depending  upon  the 
system  failure  specifications. 

A  measurement  of  the  effect  on  perform¬ 
ance  caused  by  a  fault. 

That  group  of  modules  to  which  a  fault 
is  isolated. 

Those  conditions  that  are  likely  to 
cause  functional  failures  if  a  main¬ 
tenance  action  is  not  performed. 

That  amount  of  time  required  to 
identify  and  detect  a  fault. 

Any  fault  that  will  cause  measurable 
degradation  in  performance  of 
any  function  within  the  system. 

No  measurable  affect  on  system 
performance . 

That  subfunction  that  treats  functions, 
subfunctions  and/or  entire  electronic  units 
as  testable  entities  to  be  observed  (tested). 
Known  and  quantified  inputs  are  injected  into 
the  entities  being  tested  and  the  entities' 
response  to  those  inputs  is  observed  for 
purposes  of  determining  the  integrity  of  the 
entity  under  test. 

Those  designs  that  are  in  support  of 
PM/FD/FL  requirements. 


Structured  Methodology  That  design  which  divides  functions 

into  separate  sub-functions  that  may  be 
designed/ tested/measured  separately. 

Fault  Localization  That  function  which  further  isolates  faults 

found  by  the  performance  monitoring  function 
and/or  fault  detection  function,  down  to  a 
Fault  Isolation  Group  (FIG)  to  allow  for  a 
maintenance  action. 

Usually  employs  more  comprehensive  tests  and 
provides  greater  fault  isolation  than  the 
performance  monitoring  or  fault  detection. 

Usually  performed  while  the  system  or 
particular  function  under  test,  is  off  line 
as  the  tests  are  usually  invasive  and  normal 
operation  of  the  unit  under  test  would  not  be 
possible . 

That  function  which  performs  extensive  tests 
to  find  faults  or  to  test  if  a  maintenance 
action  has  eliminated  the  cause(s)  of  faults. 


System  Integrity  That  state  of  readiness  where  all 

functions  and  all  monitored  points  indicate 
that  the  functions  meet  all  performance 
requirements  and  that  no  faults  exist. 

White-box  Methodology  That  design  which  states  that  by  dividing  a 

system  into  separate  blocks,  the  individual 
blocks  will  have  internal  test  points  that 
will  adequately  provide  a  statement  as  to 
the  block's  integrity. 

Black-box  Methodology  That  methodology  which  treats  entire  entities 

or  portions  of  electronic  units  as  testable 
entities  to  be  observed  (tested).  Known  and 
quantified  inputs  are  injected  into  the 
entities  being  tested  and  the  entities 
response  to  those  inputs  is  observed  for 
purposes  of  determining  the  integrity  of  the 
entity  under  test. 
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APPENDIX  C 

PM/FD/FL  SAMPLE  PRESENTATION  SLIDES 
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PM/FD/FL 

ARE  METHODS  OF  ENSURING  SYSTEM  INTEGRITY  AND  CONSISTANCY  TO  ELIMINATE 
THE  NEED  FOR  THE  OPERATOR  TO  MAKE  VALUE  JUDGMENTS. 

THE  SYSTEM  INTEGRITY  IS  NOT  DETERMINED  BY  THE  OPERATORS'  MOTIVATION, 
TRAINING,  OR  ABILITY. 

THE  SYSTEM  INTEGRITY  IS  DETERMINED  BY  THE  DESIGN  OF  THE  PM/FD/FL 
SYSTEM 

THE  DESIGN  AND  REPEATABILITY  OF  THE  PM/FD/FL  SYSTEM  SHOULD  BE  GIVEN 


HIGH  PRIORITY. 


WHY  ALL  THE  FUSS  ABOUT  PM/FD/FL?? 

BECAUSE  IT  IS  A  FIRST  STEP  TOWARDS  PROVIDING  NON-STOP  ELECTRON 
SYSTEMS . 

TODAY  MORE  AND  MORE  ELECTRONIC  SYSTEMS  MUST  HAVE  A  HIGH  CONFID 
LEVEL . 

TODAY'S  HIGHLY  SOPHISTICATED  ELECTRONIC  SYSTEMS  ARE  SO  COMPLEX 
NO  ONE  INDIVIDUAL  COULD  BE  EXPECTED  TO  KNOW  THE  ENTIRE  SYSTEM. 


IF  A  SYSTEM  IS  FREE  FROM  ERROR  98%  OF  THE  TIME  AND  YOU  GET  A  RESPONSE, 
HOW  DO  YOU  KNOW  IF  THE  RESPONSE  IS  CORRECT  OR  IN  THE  2%  ERROR  MARGIN? 

AT  PRESENT  PM/FD/FL  IS  NOT  A  PANACEA  BUT  IT  IS  AN  APPROACH  TOWARD 
IMPROVEMENT  OF  CONFIDENCE  LEVELS  AND  RESPONSE  INTEGRITY. 

IT  IS  TRULY  A  THIRD  GENERATION  APPROACH  TO  COMPUTER  RELIABILITY. 


PM/FD/FL  IMPLIES  ON-LINE  OPERATION 

THE  CYCLIC  RATE  OF  PM/FD/FL  TESTING  IS  CONSTRAINED  BY  THE 
COMPUTER  ARCHITECTURE 
LOGIC  DESIGN 

RESIDENT  EXECUTIVE  OPERATING  SYSTEM 

SOFTWARE  LANGUAGE 

SOFTWARE  SPEED  REQUIREMENTS 

IDEALLY  THE  PM/FD/FL  SYSTEM  SHOULD  BE  RUN  AS  NON-INVASIVE  AS 
POSSIBLE.  EVEN  WHERE  THIS  IS  NOT  POSSIBLE,  IT  SHOULD  RUN  ON 
ALGORITHM  PREDETERMINED  NOT  BY  AN  OPERATOR,  BUT  RATHER  BASED 
UPON  THE  PRIORITY  NEEDS  OF  EACH  OF  THE  SUB-SYSTEMS  WHICH  MAKE 
UP  THE  TOTAL  SYSTEM. 
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PM/FD/FL 


O  STOPPING  THE  NORMAL  RUNNING  SYSTEM  FOR  AN  ADVANCED  PM/FD/FL 
CHECK  SHOULD  BE  POSSIBLE;  BUT  THE  MORE  MANUAL  THE  METHOD,  THE 
LESS  EFFECTIVE  THE  DESIGN  PHILOSOPHY  OF  PM/FD/FL. 


O  THE  WHOLE  DESIGN  CRITERIA  FOR  PM/FD/FL  IS  TO  ALLOW  FOR  CONSISTANCY 
IN  COMPUTER  INTEGRITY  SANS  THE  COMPUTER/TERMINAL  OPERATOR. 
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PERFORMANCE  MONITORING 

PERFORMANCE  MONITORING  IS  PROBABLY  THE  LEAST  UNDERSTOOD  AND  MOST  MIS¬ 
USED  SUBSYSTEM  OF  PM/FD/FL. 


► 


O  THE  PM  SUBSYSTEM  IS  NOT  THE  SAME  AS  THE  FAULT  DETECTION  SUBSYSTEM. 

O  PM  IS  NOT  THE  CONVERSE  Or  FAULT  DETECTION;  I.E.  FAILURE  TO  DETECT 
A  FAULT  IS  NOT  PRIMA  FACIE  EVIDENCE  THAT  THE  PERFORMANCE  MONITORING 
SYSTEM  IS  WORKING  CORRECTLY. 

O  PERFORMANCE  MONITORING  IS  A  MACRO  MEASUREMENT  OF  THE  HEALTH  OF  THE 
SYSTEM. 

O  PERFORMANCE  MONITORING  IS  TO  DETERMINE  SYSTEM  INTEGRITY  BY  TREATING 
THE  ENTIRE  SYSTEM  AS  A  "BLACK  BOX"  IN  THAT  PREDETERMINED  INPUTS 
SHOULD  GIVE  CALCULATABLE  OUTPUTS. 
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PERFORMANCE  MONITORING  (CONTINUED) 


PERFORMANCE  MONITORING  IS  NECESSARY  BECAUSE: 

1.  TOLERANCES  OF  SUBSYSTEMS  WITHIN  THE  LARGER  SYSTEM 
COULD  BE  WITHIN  THEIR  INDIVIDUAL  ACCEPTABLE  TOLERANCES 
YET  THOSE  TOLERANCES  CAN  ADD  UP  IN  SUCH  A  WAY  AS  TO 
DEGRADE  THE  MAJOR  SYSTEM. 

2.  THE  FAULT  DETECTION  SYSTEM,  IN  A  PRACTICAL  SENSE,  CANNOT 
TEST  EVERYTHING. 


3.  TRENDS  AND  TENDENCIES  TO  AN  EVENTUAL  FAILURE  WOULD  LIKELY 
SHOW  UP  FIRST  AS  A  PERFORMANCE  MONITORING. 


4.  THE  "WHOLE"  IS  GREATER  THAN  THE  SUM  OF  ITS  PARTS. 


PERFORMANCE  MONITORING  (CONTINUED) 


PERFORMANCE  MONITORING  SHOULD: 

NOT  BE  UNDER  OPERATOR  CONTROL. 

BE  RUN  INDEPENDENTLY  AS  AN  INTERACTIVE  PROCESS  WHERE 
NECESSARY 

BE  RUN  AS  A  CONCURRENT  PROCESS  IF  AT  ALL  POSSIBLE. 

BE  RUNNING  AT  ALL  TIMES  WHEN  THE  SYSTEM  IS  POWERED  UP  AND 
INITIALIZED. 

THE  OPERATOR  SHOULD  BE  INFORMED  AS  TO  THE  PERFORMANCE  OF  THE 
SYSTEM  (E.G.,  "ALL  MONITORED  SYSTEMS  AND  DATA  IN  THE  PM  SYSTEM 
ARE  SATISFACTORY  AT  THIS  TIME"). 
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FAULT  DETECTION 


THE  FD  PART  OF  PM/FD/FL 


THE  PURPOSE  OF  FAULT  DETECTION  IS  TO  DETECT  FAULTS  THAT  OCCUR  IN  A 


SYSTEM.  THE  DETECTION  PROCESS  IS  USUALLY  DESIGNED  USING  THE  "WHITE 


BOX"  METHOD.  IN  OTHER  WORDS,  EACH  MODULE  WITHIN  A  SYSTEM,  IS 


SCRUTINIZED  IN  ORDER  TO  DECIDE  WHAT  CRITERIA  (PECULIAR,  PROBABLY,  ONLY 


TO  THAT  MODULE)  WOULD  BEST  ENSURE  RELIABLE  FAULT  MONITORING. 
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FAULT 


A  FAULT  IS  WHEN: 

+  ANY  PORTION  OF  EQUIPMENT,  OR  PROGRAM  DOES  NOT 
PERFORM  AS  COULD  REASONABLY  BE  EXPECTED. 

+  A  FAULT  MAY  OR  MAY  NOT  BE  DETECTABLE. 

+  A  FAULT  MAY  OR  MAY  NOT  RESULT  IN  SYSTEM  FAILURE 
+  A  FAULT  OCCURS  WHENEVER  ANYTHING  DOESN'T  WORK. 

+  FAULT  DETECTION  IS  CONSIDERED  PART  OF  THE  "FD" 

SUBSYSTEM,  AND,  ALTHOUGH  THERE  IS  A  RELATION  TO 
THE  PERFORMANCE  MONITORING  SUB-SYSTEM,  IT  IS 
SEPERATE  BY  DESIGN,  BY  PLAN,  AND  BY  OBJECTIVE. 

FAULTS  ARE  USUALLY  CLASSIFIED  AS  TO  MAJOR  (FATAL  AND  UNRECOVERABLE), 
MINOR  (WILL  ONLY  DEGRADE  PERFORMANCE)  AND  RECOVERABLE  (SYSTEM  WILL 
STILL  FUNCTION  AS  PLANNED  AFTER  SOME  ACTION  TAKES  PLACE). 

FAULT  DETECTION  DESIRABLY  SHOULD  BE  ON-LINE  AT  ALL  TIMES.  THAT  IS, 
THE  FAULT  DETECTION  SYSTEM  SHOULD  RUN  CONTINUOUSLY  ON  ITS  OWN  WITHOUT 
OPERATOR  INTERVENTION. 

FAULT  DETECTION  THEN  IS  FREE  FROM  OPERATOR  ABILITY  AND  ITS  INTEGRITY 
IS  DETERMINED  BY  THE  DESIGN  PHILOSOPHY. 
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PM/FD/FL  PHILOSOPHY 


FD  IS  PERFORMED  AUTOMATICALLY  ON-LINE  AND  PROVIDES  SUPPORT 
FOR  ISOLATION  OF  FAULTS  TO  A  UNIT  LEVEL. 


FD  IS  ALSO  PERFORMED  ON  A  SCHEDULED  OFF-LINE  BASIS. 


FL  IS  PARTIALLY  PERFORMED  BY  ON-LINE  FD. 
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APPENDIX  D 
APPLICABLE  DIDS 
FOR  TASKS 


37/58 
Reverse  Blank 
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ESTABLISHING  PM/FD/FL  CONTRACTURAL  REQUIREMENTS 

TABLE  1 

COMPARISON  OF  PM/FD/FL  PROGRAM  TASKS 
PM/FD/FL  CDRL  REQUIREMENTS 

PROGRAM 


PROGRAM  PLAN 
DESIGN  REVIEWS 
MODELING 
ALLOCATION 
PREDICTION 
FAULT  TREE 

FAULT  IDENTIFICATION 
FAULT  CERTIFICATION 
INDEPENDENT  VERIFICATION/ 
VALIDATION 

CONFIGURATION  MANAGEMENT 
FAULT  IMPACT 
TRANSIENT  SMOOTHING 


Also  included  for  possible  use  are 

DI-R-7105  DATA  COLLECTION 

DI-T-7198  TESTABILITY  PROGRAM  PLAN 

DI-T-7 199  TESTABILITY  ANALYSIS  REPORT 

DI-ATTS-XXA  (TAILORED)  PM/FD/FL  MODELING  REPORT 

DI-ATTS-XXXB  (TAILORED)  PM/FD/FL  ALLOCATION  REPORT 

DI-ATTS-XXXC  (TAILORED)  PM/FD/FL  PREDICTION  REPORT 
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DATA  ITEM  DESCRIPTION 

I  “iTLi 

hardware  diagnostic  test  system  development  plan 


l.  Q£SCa:PT.OMPuAPOS€ 


?Qtm  AoorovrG 
CMS  Wo.  3/OA-OfM 
f->o.  Car  jun  jo  1966 
2.  OeNTIfiCAriON  NUMBER  — - ■ ““ 

DI-AT7S- 80005 


3.1  The  Hardware  Diagnoscic  Test  System  (HDTS)  Development  Plan  describes  the  con¬ 
tractor's  plan  £or  developing  and  integrating  a  hardware  fault  diagnostic  and  test 
capability  for  system/sub  system/ equipment .  It  provides  a  controlled  statement  of  the 
contractor's  plan  for  producing  and  developing  the  diagnoscic  software  and  hard¬ 
ware  diagnoscic  test  devices  which  satisfy  the  functional,  performance,  and 


4  APPROVAL  OATS 

$  OffiCB  Of  PRIMARY  RESPONSIBILITY  (OPR) 

Sa.  OTIC  REQUIRED 

66.  GIOER  REQUIRED 

(YYMMOO) 

G/T213 

850610 

7  APPLICATION /INTERRELATIONSHIP 


7.1  The  Hardware  Diagnoscic  Tesc  System  Development  Plan  provides  the  contractor 
with  the  means  to  coordinate,  control,  and  monitor  progress  of  the  development  ef¬ 
fort.  It  provides  che  Government  wich  knowledge  of  the  schedule,  organization  and 
resource  allocation  planned  by  the  contractor.  It  is  a  basic  tool  with  which  che 
Government  car.  monitor  che  contract  work  efforc. 


7.2  This  daca  item  description 
DOD-STD-170 1 (NS) 

a.  APPROVAL  LIMITATION  “  ~  — 


(DID)  satisfies  Che  requirements 

(9a.  APPLICABLE  fQRMS 


of  paragraph  5.1, 


96  AMSC  NUMBER 


G3611 

10.  PREPARATION  INSTRUCTIONS  ’ 


10-1  Source  document.  This  applicable  issue  of  the  document  cited  herein,  including 
its  approval  dace  and  daces  of  any  applicable  amendments  and  revisions,  shall  be  as 
reflected  in  che  contract. 

10.2  The  HDTS  development  plan  shall  consist  of  ten  sections  with,  appropriate 
subsections.  The  format  shall  be  as  follows. 


Section 

I 

-  Introduction 

Section 

II 

-  Organization  and  Responsibility 

Section 

III 

-  Management  and  Technical  Controls 

Section 

IV 

-  Resources 

4.1  Personnel 

4.2  Training 

4.3  Data  Processing  Equipment 

Section  V 


Software  Development  Schedule 


I  •>  #.* 
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Copy  available  to  DTIC  doos  not 
permit  fully  legible  reproduction 


3.  DESCRIPTION/PURPOSE  (Cont’d) 

operational  requirements  of  the  system/subsystem/equipment .  It  is  used  to  approve 
the  contractor's  approach  for  a  Hardware  Diagnostic  Test  System  (HDTS),  and  to  mon¬ 
itor  and  evaluate  the  contractor's  progress  while  developing  the  HDTS. 

10.  PREPARATION  INSTRUCTIONS  (Cont'd) 

Section  VI  -  Monitoring  and  Reporting 

Section  VII  -  Documentation 


Section  VIII 


Section  IX 


Section  X 


-  Development  Approach 

8.1  Engineering  Practices 

8.2  Operating  Practices 

-  Development  and  Test  Tools 


-  Security  Controls  and  Requirements 


10.3  The  content  of  each  section  shall  be  as  follows. 

10.3.1  Section  I.  Introduction.  This  section  shall  describe  the  scope,  purpose, 
application  and  authority  of  the  development  effort.  This  should  include  a  brief 
overview  of  the  management  philosophy  and  methodology  that  will  be  used  on  the 
project. 

10.3.2  Section  II.  Organization  and  Responsibility.  This  section  shall  describe 
Che  organization,  responsibilities  and  structure  of  the  groups  that  will  be  design¬ 
ing,  producing  and  testing  all  segments  of  the  software  system.  It  shall  also 
identify  the  name  and  management  position  of  each  supervisor. 

10.3.3  Section  III.  Management  and  Technical  Controls.  This  sections  shall 
describe  the  management  and  technical  controls  that  will  be  used  during  development, 
including  controls  for  insuring  that  all  performance  and  design  requirements  have 
been  identified  and  implemented. 

L0.3.4  Section  IV.  Resources. 

10.3.4.1  Personnel.  This  section  shall  identify  the  level  of  manpower  allocated  to 
each  cask  shown  in  the  development  schedule,  including  numbers,  duration  of  as¬ 
signment,  and  required  skills.  This  includes  administrative  and  logistic  support 
personnel.  If  known,  personnel  assigned  to  software  development  tasks  shall  be 
listed  by  name.  This  section  shall  also  identify  security  clearance  requirements  and 
plans  for  obtaining  the  necessary  security  clearances  for  personnel  working  on  the 
software  svseem  (if  applicable). 

10. 3. -.2  Training.  This  section  shall  identify  training  required  for  people  v  ?  r .  r  r 
"r.  t.ne  project  ar.d  dates  by  which,  the  training  must  be  completed. 
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10.  PREPARATION  INSTRUCTIONS  (Cont'd) 

10.3.4.3  Data  Processing  Equipment.  This  section  shall  identify  requirements  for 
the  use  of  data  processing  equipment  to  support  the  development  of  computer  programs 
and  their  subsequent  testing.  It  shall  also  describe  the  plan  for  assuring  that  the 
necessary  hardware  is  available  at  the  appropriate  times. 

10.3.5  Section  V.  Software  Development  Schedule.  This  section  shall  present  a 
graphic  and  narrative  description  of  the  scheduled  events  and  milestones  of  the  soft¬ 
ware  development  effort.  The  schedule  will  be  updated  to  reflect  additional  detail 
as  the  project  moves  through  successive  phases  of  the  development  cycle.  By  Pre¬ 
liminary  Design  Review,  this  section  shall  include  a  development  schedule  for  each 
computer  program  and  data  base.  The  graphic  description  shall  be  a  chart  identifying 
schedules  for  the  following: 

a.  All  deliverables; 

b.  Preparation  of  management  and  test  plans; 

c.  All  levels  of  testing; 

d.  Reviews,  including  major  reviews  and  other  internal  milestones; 

e.  Transition  to  life-cycle  support  activity. 

The  chare  should  illustrate  a  relationship  with  hardware  schedules.  Critical 
paths  shall  also  be  identified. 

10.3.6  Section  VI.  Monitoring  and  Reporting.  This  section  shall  describe  the  pro¬ 
cedure  for  monicoring  and  reporting  the  status  of  program  development.  It  shall  also 
describe  the  manner  in  which  problems  and  recommended  solutions  to  problems  will  be 
reported . 

10.3.7  Section  VII.  Documentation.  This  section  shall  describe  the  approach  for 
developing  computer  program  documentation  and  will  identify  Che  documentation  that 
will  be  produced.  This  shall  include  the  plan  for  developing  tesc-planr.ing  documen¬ 
tation,  che  Software  Requirements  Specification,  the  Sys tem/Subsys tern  Specification, 
the  Program  Specification,  Software  Manuals  and  any  oche’-  documentation. 

10.3.8  Section  VIII.  Development  Approach. 

10.3.8.1  Engineering  Practices.  This  section  shall  describe  me  engineering  prac¬ 
tices  that  will  be  applied  to  the  development  of  software.  These  practices  include 
standards,  conventions,  procedures,  rules  for  programming,  design  and  ocher  disci¬ 
plines  affecting  development.  Ac  a  minimum,  procedures  for  implementing  che  follow¬ 
ing  practices  shall  be  described: 


a.  Programming  and  data  base  standards; 
d .  I ?p-c own  design  methodology ; 


design  wallt-thr  oughs  . 
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PREPARATION  INSTRUCTIONS  (Cont'd) 

10.3.8.2  Operating  Practices.  This  section  shall  describe  the  operating  practices 
that  will  be  applied  to  the  development  of  software.  These  include  the  following; 

a.  Use  of  Unit  Development  Folders; 

b.  Techniques  for  ensuring  that  all  performance  and  design  requirements  have 
been  implemented; 

c.  Means  of  ensuring  modularity,  ease  of  modification,  and  capacity  for  com¬ 
puter  program  growth; 

d.  Methods  and  procedures  for  collecting,  analyzing,  monitoring  and  reporting 
on  the  timing  of  Cime-critical  computer  programs; 

e.  Means  for  ensuring  that  the  software/data  processors/peripheral  equipment 
interfaces  are  adequate; 

f.  Criteria  for  determining  when  a  development  unit  should  be  entered  into 
configuration  control; 

g.  Means  of  controlling  master  copies  of  computer  programs,  data  bases  and 
associated  documentation  during  development  (including  their  relationship  to  che  Con¬ 
figuration  Management  Plan) ; 

h.  Rules  for  interface  definition. 

10.3.9  Section  IX.  Development  and  Test  Tools.  This  section  shall  identify  the 
special  cools  and  techniques  that  will  be  used  during  development  and  testing  of  the 
computer  programs.  Some  examples  are  as  follows: 


a.  Special  simulation; 

b.  Data  reduction; 

c.  Code  optimizers; 

d.  Code  audicors; 

e.  Special  utility  programs; 

f.  Software  security  test  tools. 

10.3.10  Section  X.  Security  Control  aud  Requirements.  This  section  shall  identify 
securicy  controls  that  will  be  used  during  software  development  (e.g.,  physical 
security,  document  access  controls,  computer  access  controls,  etc...  Lt  shall  also 
describe  the  method  cf  implementing  and  maintaining  the  security  tmcrols.  It  mall 
also  identifv  mi  unique  security  problems  md  installation  securitv  requ  i  re  mo:-.:  - . 
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OATA  ITEM  DESCRIPTION 


OCMTiriCATIOM  HOIS) 


i  nr^f 

DESIGN  REVIEW  DATA  PACKAGE 


OCtCHlHTlOM/HuHHOSC 

3.1  The  daca  packages  are  required  by  ehe  Government  to 
permit  adequate  preparation  for  each  design  review  prior 
to  the  review  meeting. 


NSA 

DI-E-5423 

4.  APPROVAL 

1977  Mav 

OAT| 

02 

«PPue«nON/INT|JIIICL  ATION1M 

7.1  To  be  used  on  contracts  which  require  formal  technical 
reviews  and  audics. 


MIL-STD-1521 


10  HHK*  AHATtON  IMITHuC  TlQNl 

10.1  Data  packages  shall  be  provided  for  design  review  meetings  to  be  held  on  the 
program  and  submitted  as  indicated  on  DD  Form  1423.  The  data  packages  shall  be 
designed  to  provide  adequate  preparation  Information  for  design  reviews  organized 
in  accordance  with  MIL-STD-1521  and  Appendices  B,  C,  D,  and  G.  The  detail  concents 
of  each  package  shall  Include,  but  not  be  limited  to,  the  material  -»quired  for  the 
subject  design  review,  an  agenda,  and  a  status  of  pertinent  (if  any)  action  Items 
from  previous  design  reviews  or  other  meetings. 


OATA  ITEM  DESCRIPTION 


Maintainability  Modelling  Report  , 

t 


9.  OCICmPTlON/PURPOU 

3.1  To  describe  and  show  the  development  of  a  maintain¬ 
ability  model  for  making  numerical  maintainability  ap¬ 
portionments  to  various  functions  and  levels  of  hardware 
throughout  an  item  (system,  subsystem,  equipment)  and  to  ‘ 
evaluate  the  maintainability  of  an  item  based  on  its 
maintainability  design  characteristics. 


.  APPLI C  AT IO H/ 1 N T ERNCW  AT  lOHlMlP 

7.1  This  DID  satisfies  the  data  requirement  of  para 
201.2  in  Task  201  of  MIL-STD-470A.  This  DID  is  applic¬ 
able  to  contracts  which  contain  the  requirements  for  Task 
201  "Maintainability  Modelling"  of  MIL-STD-470A. 
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*  Ml L-STD-470A 
MIL-STD-847 


I  MCIL  MUMRCRIU 

I  0MB  Exempt 
*AMSC  No.  F3216 


lN|TRUCT:ONl 


10.1  Unless  otherwise  stated  in  the  solicitation,  the  effective  date  of  the  docu¬ 
ment^)  cited  in  this  block  shall  be  that  listed  in  the  issue  of  the  DoD  Index  of 
Specifications  and  Standards  (DoOISS)  and  the  supplements  thereto  specified  in  the 
solicitation  and  will  form  a  part  of  this  Data  Item  Description  to  the  extent  defined 
within. 

10.2  The  Maintainability  model (s)  shall  be  developed  in  accordance  with  paragraph 

201.2  of  Task  201  "Maintainability  Modelling"  of  MIL-STD-470A  as  tailored  to  the  partic¬ 
ular  needs  of  the  acquisition  program. 

10.3  Format.  The  plan  shall  be  prepared  in  accordance  with  MIL-STD-847. 
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DATA  ITEM  DESCRIPTION 


1.  TITUi 

Maintainability  Allocations  Report 


lOENTlffCATfON  NO(S» 


GENCY 


DOD 


1.  AP  P  no  v  At_  o  A  T  E 


.  OCICRIPTION/PUNPOII  I 

3.1  To  document  the  Quantitative  maintainability  requirements  1-83  uanuary 
developed  for  each  component  item  of  the  approved  hardware 
breakdown  structure  derived  to  meet  the  end  item  requirements . 


7.  APPUlCATION/INTIRPUt  ATlONJHlP 

7.1  This  010  satisfies  the  data  requirements  of  para  202.2  in 
Task  202  of  MIL-STD-470A.  System/Subsystem/equipment  level 
quantitative  maintainability  requirements  must  be  broken  down  to 
appropriate  subsystem/equipment/unit/subunit  levels  as  neces¬ 
sary  to  establish  requirements  for  designers  and  subcontractors . 
This  DID  is  applicable  whenever  Task  202  "Maintainability  Allo¬ 
cation"  of  MIL-STD-470A  is  called  out  as  part  of  an  acquisition 
program. 

7.2  Tins  DID  supersedes  UDI-R-23570. 
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10.1  Unless  otherwise  stated  in  the  solicitation,  the  effective  date  of  the  document (s ) 
cited  in  this  block  shall  be  that  listed  in  the  iss^e  of  the  DcD  Index  of  Specifications 
and  Standards  (DoDISS)  and  the  supplements  thereto  specified  in  the  solicitation  ana  will 
form  a  part  of  this  Data  Item  Description  to  the  extent  defined  within. 

10.2  Maintainability  Allocations  reports  shall  be  prepared  in  accordance  with  paragraph 

202.2  of  Task  202  "Maintainability  Allocation"  of  MIL-STD-470A  as  tailored  for  the'par- 
ticular  acquisition.  The  report  shall  provide  the  results  and  describe  the  process  cf 
allocating  Maintainability  requirements  to  each  component  end  item. 

10.3  Format :  The  report  shall  be  prepared  in  accordance  with  MIL-STD-847. 
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DATA  ITEM  DESCRIPTION 


<•  TITLC 

Maintainability  Predictions  Report 


S.  OtSCRlPTION/PURPOtl 

3.1  The  description  and  documentation  of  the  maintainability 
prediction  made  by  the  contractor.  To  make  a  determination  of 
whether  or  not  the  proposed  design  is  consistent  with  maintain¬ 
ability  requirements. 


IDENTIFICATION  NOiSI 


AGENCY 

NUMBER 

DOD 

DI-R-71 08 

4.  APPROVAL, 

O  ATC 

1983  January  3 

#.  okw ice  ok  PRtMAnv 
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AP  PLICATION/ IN  ▼  CRN  Cl.  AT  l  ON  SHIP 


7.1  This  DID  satisfies  data  requirements  of  para  203.2  in  Task 
203  of  MIL-STD-470A.  Performance  of  Maintainability  predictions 
applicable  to  Task  203  "Maintainability  Predictions"  of  MIl-STD- 
470.  The  content  of  this  report  shall  be  included  in  the  "Main¬ 
tainability  Prediction  Report"  of  MIL-HD8K-472  when  that  has 
been  designated  as  the  basis  for  Task  203  of  MIL-STD-470A. 

7.2  This  DID  supersedes  DI-R-2128. 


•  A  ZW  CP  CNC  C3  (SimnOMtory  < 
block  10 ) 

*  MIL-STD-470A 
MIl-STD-347 
MIL-HDBK-472 


MCSU  NUMItNIII 

0“8  Exempt 
*AMSC  No.  F3216 


10.  PM  CP  A  PAT  ION  INSTRUCTIONS 

10.1  Unless  otherwise  stated  in  the  solicitation,  the  effective  date  of  the  document(s) 
cited  in  this  block  shall  be  that  listed  in  the  issue  of  the  DoD  Index  of  Specifications 
and  Standards  (DoDISS)  and  the  supplements  thereto  specified  in  the  solicitation  and  will 
form  a  part  of  this  Data  I<  3m  Description  to  Lhe  extent  defined  within. 

10.2  Maintainability  Predictions  Report  shall  be  prepared  in  accordance  with  paragraph 

203.2  of  Task  203  of  MIL-STD-470A  as  tailored  for  the  particular  acquisition. 

10.3  The  maintainability  predictions  report  shall  contain  such  detail  as: 

a.  assumptions  used  in  tne  prediction  process 

b.  identification  of  the  prediction  procedure  used 

c.  prediction  results  to  the  appropriate  item  levels. 

10.4  Format :  The  report  shall  be  prepared  in  accordance  with  MIl-STD-347. 


THIS  DOCUMENT  CO^TAINo 


PAGES. 


DD -22**..  1664 

1811 


O  *  ,.-u  O  ACC  9 


-NT  PRINTING  OPF'CE  1983— 605  033-9030 


+  c  *“ ,  A1;  X 


*Wjt 

M 


h *11  ,V  Kh 


,'iVi 


DATA  ITEM  DESCRIPTION 
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2.  lOSMTlFtCAftON  lUMtU 


DI-MISC-80048 


Scientific  and  Technical  Reports  Summary 


i.  ojsoumcM/wwose 

3.1  Technical  reports  are  acquired  to  provide  the  scientific  and  technical 
community  a  description  of  the  precise  nature  and  results  of  research,  develop¬ 
ment,  test,  and  evaluation  (RDT4E)  accomplished.  Technical  reports  may  be  defin¬ 
itive  for  the  subject  presented,  exploratory  in  nature,  or  an  evaluation  of  criti¬ 
cal  subsystem  or  of  technical  problems. 


4.  approval  OAti 
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7  APPLICATION /INTtRRJUAnONSHIP 

7.1  This  Data  Item  Description  contains  the  data  format  and  content  preparation 
instructions  for  the  data  product  generated  by  the  specific  and  discrete  task 
requirements  for  this  data  included  in  the  contract. 

7.2  This  Data  Item  Description  shall  be  used  in  preparing  all  ongoing  interim  or 
final  Scientific  and  Technical  Reports  Summary.  The  purpose  of  these  report  sum¬ 
maries  is  to  present  management  with  a  concise  description  of  the  scientific  and 
technical  findings  and  accomplishments  during  the  reporting  period. 


».  APPROVAL  LIMITATION 


.  APPUCAIU  FORMS 

*3.  A MIC  NUMIEI 

A3  6  70 

10.  PREPAAAnON  INSTRUCTIONS 

10.1  Contract.  This  Data  Item  Description  is  generated  by  the  contract  which 
contains  a  specific  and  discrete  work  task  to  develop  this  data  product. 

10.2  Format.  The  Scientific  and  Technical  Reports  Summary  shall  be  in  contractor 
format. 

10.3  Contents .  The  level  of  detail  of  the  Scientific  and  Technical 
Reports  Summary  shall  be  adequate  for  non-specialists  in  the  subject  matter. 

When  appropriate,  specific  references  should  be  made  to  more  detailed 
materials.  The  content  of  the  Scientic  and  Technical  Report  Summary  shall 
consist  of  the  following: 

(a)  Task  objectives. 

(b)  Technical  problems. 

(c)  General  methodology  (e.g.,  literature  review,  lab  experiment, 
survey ,  etc) . 

id)  Technical  results. 

i  e  '•  Important  findings  and  ::nclusions. 
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Scientific  and  Technical  Reports  Summary  (Cont'd) 

Block  7  APPLICATION /INSTRUCT IONS  (Cont'd) 

7.2  (Cont'd)  The  types  of  scientific  and  technical  report  summaries  and  their 
frequencies  are  specified  in  the  DD  Form  1423 

7.3  This  Data  Item  Description  shall  be  applicable  in  contracts  when  DI-S-4057 
is  used. 


Block  10  PREPARATION  INSTRUCTIONS  (Cont'd) 

10.3  (Cont’d) 

(f)  Implications  for  futher  research 

(g)  Significant  hardware  development 

(h)  Special  comments 

10.4  Cover  Page  -  The  heading  or  cover  page  of  each  report  summary  shall 
contain  the  following  information: 

(a)  Procuring  Activity  Designated  Order  Number 

(b)  Name  of  Contractor 

(c)  Contract  Number 

(d)  Effective  Date  of  Contract 

(e)  Expiration  Date  of  Contract 

(f)  Reporting  Period 

(g)  Principal  Investigator  and  Phone  No. 

(h)  Project  Scientist  or  Engineer  and  Phone  No 

(i)  Short  Title  of  Work 

10.4.1  Additionally,  each  report  produced  will  have  prominently  displayed  on  the 
cover  page,  a  notice  of  disclaimer  worded  as  follows: 

The  views  and  conclusions  contained  in  this  document  are  those  of  the 
authors  ar.d  should  not  be  interpreted  as  necessarily  representing  the  offioal 
policies,  either  expressed  or  implied,  of  the  Government. 

’  0 . .  2  Scientific  and  Technical  reports  which  are  spc.nstred  tv  otrer  than  the  or" 
curing  activity  shall  nave  the  following  or.  the  front  cover: 


8 

£ 

$ 

g 


-y 'v-v-w .v-.v v.v v.v.v. 


mgmtg vmrnsxm  ebb  mwu  'jj  w  ■  ji  pji  'jj  ?  wr  ■>  ■:»’-•  rrA’-’v.vv-’T'.v.r./,  ^  v; 


DI-MI SC-80048 

Scientific  and  Technical  Reports  Summary  (Cont'd) 

Block  10  PREPARATION  INSTRUCTIONS  (Cont'd) 

Sponsored  by 

(Sponsor's  Identification) 

(Sponsor's  Designated)  Order  No.  _ 

Monitored  by  _ 


Under  Contract# 


10.5  Reports  shall  be  reproduced  only  by  processes  which  provide  black  on  white 
copy  sufficiently  clear  and  sharp  for  further  reproduction  when  required.  Ditto, 
hectograph,  color,  and  other  reproduction  processes  not  reproducible  photograph¬ 
ically  or  xerographically  are  not  acceptable. 


DATA  ITEM  OESCRIPTIO* 


PROCEDURES,  TEST 


JCIC  RATION/  RURCOK 


This  data  item  is  used  Co  describe  a  contractor's  test 
procedure  and  how  he  intends  to  determine  compliance  with 
specification  requirements. 


IDENTIFICATION  ;i:. 


NAVY-SE  UDI-T-237323 


A'  AN  m  NOV  Ac  O  A  T  ( 

74  Oct  23 

1-  OFFICE  OF  FNlUANT 
AtlFONFULITT 

SEA  9833 


AFFUCATION/INTtRFtL  ATIOMFMIN 


Application  will  be  as  specified  by  Che  contract  data 
requirements  list.  This  item  may  be  used  whenever  tests 
are  required. 


i  M9ARATIOM  IHI"  RUCTIONS 

10.1  The  test  procedures  shall  be  typed  in  contractor  or  commercial  format  on 
8"xlOV  sheets. 

10.2  The  test  procedures  shall  cover  in  detail  the  plan  and  procedures  for  ac-c::pli« 
ment  of  the  tests  specified  in  the  contract  schedule  and  specifications  reference-: 
therein  or  in  Block  16  of  the  DD  Form  1423,  Contract  Data  Requirements  List,  da*..-- 
item  requiring  these  procedures  and  shall  specifically  cover  or  contain  the  follc-vlng 
as  applicable: 

a.  Title 

b.  Index 

c.  Identification  of  item  being  tested  (serial  number) 

d.  Identification  number  of  test  procedure 

e.  Hardware  configuration 

f.  Teat  prerequisites 

g.  Report  form 

h.  Date,  time  and  duration  of  test 

i.  Proposed  test(s) 

j .  Preoperational  checklist 

k.  The  purpose  of  the  test(s) 


.Tnwmw, 


UDI-T-23732B 


PROCEDURES,  TEST  (Con.  ..nued) 


l.  Description  of  test 

m.  The  specification  paragraph(s)  to  which  the  test(s)  will  prove 
compliance. 

n.  Detailed  step-by-step  procedure  (may  be  referenced  to  test  number 
and  test  title  in  Government  documents) 

o.  Test  schedule  (operating  profile,  setpoints,  stabilization  time, 
data  points) 

p.  The  test  equipment  utilized. 

q.  Approvals,  authorities  and  responsibilities 

r.  Sketches  or  photographs  of  test  set-up 

s.  Facilities  required  for  test 

t.  Tesc  equipment  requirements  (major  and  special) 

u.  Methods  of  measurement (s) 

v.  Logistics  equipment  requirements  (spare  test  hardware) 

v.  Method  of  control  of  sub-contractor's  efforts  and  their  procedures. 

x.  Applied  instrumentation  and  data  recording  equipment 

y.  Data  sheets  (when  required  b7  a  specification)  for  which  the  results 
arc  able  to  be  correlated  to  the  item  tested. 

z.  Types  of  data  to  be  recorded  (parameters,  ranges,  accuracies,  type 
readout,  and  quantities) 

aa.  Results  (-omparison  of  test  data  to  acceptance  standard) 

bb.  Accept, 'r«j ect  criteria  for  test  acceptance. 

cc.  Personnel  required 

dd.  Special  resource  requirements 

ee.  Reference  to  specs,  standards,  tech  manuals,  other  test  procedures 

and  reports,  change  orders,  notices,  and  other  references  not  specific 
to  the  r  st  but  included  for  information  only. 

In  addition  c.c  the  requirements  of  paragraph  10.2,  the  production  test 

edures  s. _  cover  cleaniiv;/refurbishing  of  test  equipment  and,  if  applicable, 

rrclartonship  for  and  during  availability  test(s). 
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APPROVAL  DAT! 


26  February  1971 


MIL-STD-4S  3  (USAF) 


DATA  ITEM  DESCK1PTION 


Configuration  Management  Plan  (CMP) 


ococpiption/puppooc 

This  plan  is  prepared  by  the  contractor  to  describe  his 
assignment  of  responsibilities  organizationally  and  the 
procedures  used  in  his  accomplishment  of  the  specific 
configuration  management  requirement  as  stated  in  the 
contract.  It  is  not  to  be  used  as  a  contractual  require¬ 
ment  in  lieu  of  the  statement  of  work. 


APPUCATIQM'-NTIRMIL  ATlONlNlP 

Obtained  as  part  of  the  val  idation  phase  final  report. 

When  a  validation  phase  is  not  accomplished,  the  CMP 
will  be  a  requirement  of  the  full-scale  development  ’  6 '  '"''a‘'°'v  ‘ 

contract.  Not  to  be  used  on  follow  on  contracts  where 

the  contractor's  configuration  management  organisation  MIL-STD-4S3  (US. 
and  procedures  have  been  satisfactorily  demonstrated 
on  prior  contracts.  This  DID  may  be  modified  ard  used 
on  competitive  RFPs  to  acquire  information  for  source 
selection.  When  used  in  this  manner,  only  an  abbreviat¬ 
ed  plan  will  be  acquired.  By  the  same  token,  when  this 
plan  is  procured  (on  other  than  validation  contracts)  it 
should  be  modified  to  delete  source  selection  require¬ 
ments. 


10.  aoiaahation  INSTRUCTION! 

The  contractor  shall  describe  in  a  configuration  management  plan,  the 
organizational  responsibilities  and  procedures  '.sod  in  the  it’  plementation  of 
the  configuration  management  requirecu-ms  as  stated  in  :he  contract.  The 
conf  ig  u  ra  t  iii  n  management  plan  shall  be  prepared  in  accordance  with  the 
criteria  set  forth  in  Appendix  I  of  MIL-STD-4S3  (USAF). 
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OATA  ITEM  DESCRIPTION 


Data  Collection,  Analysis  and  Corrective  Action  System,  Reports 


OCMTiriCATlOH  HOIS' 


01  -R- 71 05 


1  DCICMI*tio,i/i 


3.1  This  data  is  used  to  aid  maintainability  design,  identify 
corrective  action  tasks  and  to  evaluate  test  results.  The  re¬ 
ports  generated  shall  consist  of  tabulations  and  analyses  of  all 
maintenance  actions  occurring  through  the  reporting  period  as 
well  as  remedial  actions  proposed  by  the  contractor  to  eliminate 
maintainability  deficiencies  (and  fault  detection/isolation  de¬ 
ficiencies  ) . 


1983  January  3 


■  CIPONti 


?  4»*UlC*tiON/iN»I««lL  iTiOwjm* 


7.1  This  DID  satisfies  the  data  requirements  of  para  104.2  in 
Task  104  of  MH-STD-470A.  This  report  is  applicable  when  Task 
104,  "Data  Collection,  Analysis  and  Corrective  Action  System"  of 
MH-STD-470A  is  called  out  as  part  of  tne  acquisition  program. 
This  010  should  be  prepared  in  conjunction  w’th  the  "Maintain¬ 
ability  Demonstration  Reports"  called  out  in  mil  -  STD- 4  7 1  a. 

7.2  This  010  Supersedes  the  following  0 IDs :  QI-R-  3537A  and  01- 
R-20665 . 
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10.1  unless  otherwise  stated  in  tne  solicitation,  the  effective  date  of  the  docu¬ 
ment',*)  cued  in  this  block  sha  ’  be  that  listed  in  the  issue  of  the  DoD  Inoe*  o ‘ 
Specifications  and  Standards  (0'JISS)  and  the  supplements  thereto  specified  in  the 
solicitation  and  will  form  a  c^rt  of  this  Data  Item  Description  to  the  extent.  defined 
within. 

10.2  The  report  confer.'  shall  descr'be  the  results  of  the  "Data  Collection,  Analysis 
and  Corrective  Action  .ystem" . 

a.  The  report,  which  may  .e  preparec  in  the  contractors  selected  format, 
shall  include  subco”  ractor,  vendor  data  as  applicable. 

b.  Outa  collected,  analyzed  and  documented  should  be  ”epresertat ' ve  of  the 
information  elements  contained  below: 

(1)  A  maintenance  event  identification  nunce” 

(2)  Maintenance  task  identif,cat-on,  ne/ec  to  eacn  ma'ntenance  e ve” t 
(detection,  isolation,  removal,  checkout,  etc.) 

;  3 ;  Oate  on  wnicn  tnp  •”a'"te”ance  e.?”t  too«  o  ' 

4'  I  dent  i  f  ’  c  at  ’  C”  c*  t”e  ’ocafcr  t"s  *  *  e.j”'  ■ 
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DI-R-7105 

10.  Preparation  Instructions  (continued) 

(5)  Identification  of  system,  subsystem,  assembly,  printed  circuit  card  on 
which  maintenance  was  performed. 

(6)  Maintenance  time  necessary  for  corrective  actions  (or  maintenance  manhours, 
where  appropriate) 

(7)  Deficiencies  found/corrective  actions  taken. 

10.3  Format:  The  report  shall  be  prepared  in  accordance  with  MIL-STD-847. 
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DATA  ITEM  DESCRIPTION 


0**«No.  0704-01** 

Ixe i  Oir»:  Jurt  JO,  IJK 
i  2.  'OfNTIftOriON  NUMliJ 


Scientific  and  Technical  Reports  Summary  DI-MISC-80048 

i.  OISC»mON/FUWOSE 

3.1  Technical  reports  are  acquired  to  provide  the  scientific  and  technical 
community  a  description  of  the  precise  nature  and  results  of  research,  develop¬ 
ment,  test,  and  evaluation  (RDT4E)  accomplished.  Technical  reports  may  be  defin¬ 
itive  for  the  subject  presented,  exploratory  in  nature,  or  an  evaluation  of  criti¬ 
cal  subsystem  or  of  technical  problems. 


[XAeeSovAt  oar* 
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7. 1  Thi3  Data  Item  Description  contains  the  data  format  and  content  preparation 
instructions  for  the  data  product  generated  by  the  specific  and  discrete  task 
requirements  for  this  data  included  in  the  contract. 

7.2  This  Data  Item  Description  shall  be  used  in  preparing  all  ongoing  interim  or 
final  Scientific  and  Technical  Reports  Summary.  The  purpose  of  these  report  sum¬ 
maries  is  to  present  management  with  a  concise  description  of  the  scientific  and 
technical  findings  and  accomplishments  during  the  reporting  period. 


1 1.  ApeeovAc  limitation 


ARRUCA»L£  KlMri 


90.  AM3C  NUM1E* 

A3670 


io.  »*i.*AjiAnoN  instructions 

|  ’0.1  Contract.  This  Data  Item  Description  is  generated  by  the  contract  which 
contains  a  specific  and  discrete  work  task  to  develop  this  data  product. 

10.2  Format.  The  Scientific  and  Technical  Reports  Summary  shall  be  in  contractor 
format. 

10.3  Contents .  The  level  of  detail  of  the  Scientific  and  Technical 
Reports  Summary  shall  be  adequate  for  non-specialists  in  the  subject  matter. 

When  appropriate,  specific  references  should  be  made  to  more  detailed 
materials.  The  content  of  the  Scientic  and  Technical  Report  Summary  shall 
consist  of  the  following: 

(a)  Task  objectives. 

(b)  Technical  problems. 

(c)  General  methodology  (e.g.,  literature  review,  lab  experiment, 
survey,  etc), 

(d)  Technical  results. 

!  ' e )  Important  findings  and  conclusions. 
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DI-MISC-80048 

Scientific  and  Technical  Reports  Summary  (Cont'd) 

Block  7  APPLICATION/INSTRUCTIONS  (Cont'd) 

7.2  (Cont'd)  The  types  of  scientific  and  technical  report  summaries  and  their 
frequencies  are  specified  in  the  DO  Form  1423 

7.3  This  Data  Item  Description  shall  be  applicable  in  contracts  when  DI-S-4057 
is  used. 

Block  10  PREPARATION  INSTRUCTIONS  (Cont'd) 

10.3  (Cont'd) 

(f)  Implications  for  futher  research 

(g)  Significant  hardware  development 


(h)  Special  comments 

10.4  Cover  Page  -  The  heading  or  cover  page  of  each  report  summary  shall 
contain  the  following  information: 


(a) 

Procuring  Activity  Designated 

Order  Number 

(b) 

Name  of  Contractor 

(c) 

Contract  Number 

(d) 

Effective  Date  of  Contract 

(e) 

Expiration  Date  of  Contract 

(f) 

Reporting  Period 

(3) 

Principal  Investigator  and  Phone  No. 

(h) 

Project  Scientist  or  Engineer 

and  Phone  No 

(i) 

Short  Title  of  Work 

10.4.1  Additionally,  each  report  produced  will  have  prominently  displayed  on  the 
cover  page,  a  notice  of  disclaimer  worded  as  follows: 

The  views  and  conclusions  contained  in  this  document  are  thcss.  of  the 
authors  and  should  not  be  interpreted  as  necessarily  representing  the  offical 
policies,  either  expressed  or  implied,  of  the  Government. 

’0.4.2  Scientific  and  Teor.ni :a  1  Reports  which  are  sponsored  by  otr.er  than  one  pro¬ 
curing  activity  3hall  have  t re  following  or.  one  fno.no  :over: 
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Scientific  and  Technical  Reports  Summary  (Cont'd) 

Block  10  PREPARATION  INSTRUCTIONS  (Cont'd) 

Sponsored  by 

(Sponsor's  Identification) 


(Sponsor's  Designated)  Order  No. 
Monitored  by  _ 


Under  Contract# 


10.5  Reports  shall  be  reproduced  only  by  processes  which  provide  black  on  white 
copy  sufficiently  clear  and  sharp  for  further  reproduction  when  required.  Ditto, 
hectograph,  color,  and  other  reproduction  processes  not  reproducible  photograph¬ 
ically  or  xerographically  are  not  acceptable. 


-j.«.  SOvfewMT  MlNrtNl  or#  ict  :  i«®3  -  *03-033/303 


..  Page  3  of  3  Pages 


r  lj.L||l|‘|i4'(a| 


DATA  ITEM  DESCRIPTION 


li  TITU* 


Testability  Program  Plan 


OUCMlPTION/PURPOiC 


3.1  This  plan  identifies  the  performing  activity  approach 
for  implementing  a  Testability  Program  in  accordance 
with  MIL-STD-2165. 


2  IDENTIFICATION  NO(S) 


OI-T-71 98 


4.  APPROV  AC  O  AT  C 

29  January  1985 

«  OPPICE  OP  PPIMARV 
RCSPONIIIILITV 

NAVY-EC 


AP>  *C  I  C  A  T  IO  N/  INTIPPIC  ATIONIMIP 


7.1  These  data  are  to  be  used  to  define  a  Testability 
Program  Plan. 

7.2  This  DID  may  be  used  for  all  electronic  system  and 
equipment  development  programs. 

7.3  This  DID  satisfies  the  data  requirements  of  Task  101 
of  MIL-ST0-21 65 . 


10  PPCPARATION  INSTRUCTIONS 


|  A.  APPROVAL  LIMITATION 


|P.  RCPCRINCES  r«»n*<ory  ••  ciled  m 
block  1 0) 


MIL-STD-2165 


ucu.  NUMsemsi 


AMSC  NO.  N3424 


10.1  The  applicable  issue  of  the  documents  cited  herein,  including  their  approval 
dates  and  applicable  amendments  and  revisions,  shall  be  as  reflected  in  the 
contract. 

10.2  Contractor's  format  is  acceptable. 

10.3  A  Testability  Program  Plan  shall  be  prepared  in  accordance  with  MIL-STD-2165, 
Task  101  and  include  the  following  elements,  with  the  range  and  depth  of 
information  for  each  element  tailored  to  the  acquisition  phase: 

10.3.1  A  description  of  the  work  to  be  accomplished  for  each  testability  task 
included  in  the  contractual  requi rements . 

10.3.2  The  time  phasing  of  each  task  and  its  relationship  to  other  tasks, 
particularly  maintainability  tasks. 

10.3.3  Identification  of  a  single  organizational  element  within  the  performing 
activity  which  has  overall  responsibility  and  authority  for  implementa¬ 
tion  of  the  testability  program. 

10.3.4  Identification  of  data  interfaces  between  the  organizational  element 
responsible  for  testability  and  other  related  elements. 


)D-!“„1664 
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Testability  Program  Plan 

10.  Preparation  Instructions  (Cont1 


< 


< 


ii 


10.3.5  Identification  of  the  method  by  which  testability  requirements  will  be 
integrated  with  other  design  requirements  and  disseminated  to  design 
personnel  and  subcontractors . 

10.3.6  Identification  of  testability  design  guides  and  testability  analysis 
procedures  to  be  used. 


10.3.7 

10.3.8 

10.3.9 

10.3.10 


Description  of  procedures  for  scheduling,  conducting  and  documenting  | 

testability  design  reviews.  j 

Identification  of  testability  submissions  and  their  review,  verification  ! 

and  utilization.  * 

t 

Description  of  procedures  for  identifying  testability-related  problems  4 

and  assuring  corrective  action.  9 

Description  of  procedures  and  controls  for  assuring  that  each  subcontractor 1 s  j 

testability  practices  are  consistent  with  overall  system  or  equipment  j 

requirements.  j 
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DATA  ITEM  DESCRIPTION 


IDENTIFICATION  NOItl 


AGENCY 


NuMae* 


Testability  Analysis  Report 


00D 


DI-T-7199 


OCICRlPriON/RURPOSC 


3.1  This  report  documents  the  results  of  the  testability 
requirements,  design  and  evaluation  tasks  of  MIL-STD- 
2165. 


4.  APPHOVAi.  OATf 

29  January  1985 


f.  OFFICE  OF  PPtMAAV 
RElPOMiaiLf  T  V 


NAVY-EC 


••  OOC  RIQUIRIO 


•  APPROVAL  LIMITATION 


7  AP»LlCATlON/INT|RRfL  ATIONIHI* 

7.1  These  data  are  to  be  used  to  evaluate  the  level  of 
testability  incorporated  in  a  design. 

7.2  This  DIO  may  be  used  for  all  electronic  system  and 
equipment  development  programs. 

7.3  This  DID  satisfies  the  data  requirements  of  Tasks  201, 
202  and  203  of  MIL-STD-21 65 . 


#.  RKPCRCNCCS  (Mandatory  aa  cttad  tn 
6/ocJt  IQ) 


MIL-STD-21 65 


AMSC  NO.  N3425 

>0  MtPARATION  .N»TRUCTI0M» 

10.1  The  applicable  issue  of  the  documents  cited  herein,  including  their  approval 
dates  and  applicable  amendments  and  revisions,  shall  be  as  reflected  in  the 
contract . 

1C. 2  Contractor's  format  is  acceptable. 

10.3  The  content  of  the  Testability  Analysis  Report  shall  include  the  following: 

10.3.1  General 

10.3.1.1  A  brief  description  of  the  system's  functional  operation. 

10.3.1.2  A  brief  description  of  the  functional  operation  of  each  Item. 

10.3.1.3  A  description  of  system  maintenance  and  support  concept. 

10.3.2  Testability  Requirements  Analysis  (MIL-STD-21 65 ,  Task  201) 

10.3.2.1  Description  of  methodology  used  to  trade-off  alternative  diagnostic 
concepts,  including  varying  degrees  of  built-in  test,  automatic  test 
equipment  and  manual  test. 


0.3. 2. 2 


DD  ."P*. 


Results  of  diagnostic  trade-off$,  including  the  impact  of  each  alternative 
on  readiness,  life  cycle  costs,  manpower  and  training. 

_ =23= _ _ 

1 664  '  '  *  '■  •  •  »  4,0  »•’  *C>  «!•>••’  J  '  „  1  o.  2  ,.cl, 
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Testability  Analysis  Report 


10.  Preparation  Instructions  (Cont'd) 


10.3.2.3  Description  of  the  selected  system  diagnostic  concept  including  recommended 
testability  requirements  for  the  system  specification. 

10.3.2.4  Description  of  methodology  used  to  allocate  system  testability  requirements 
to  each  item;  recommended  testability  requirements  for  each  item. 

10.3.3  Preliminary  Testability  Design  Analysis  (MIL-STO-2165.  Task  202) 


10.3.3.1  Description  of  system  built-in  test  functional  design  and  system  partitioning 
used  to  enhance  testing. 

10.3.3.2  For  each  item  to  be  included  in  this  analysis,  a  description  of  testability 
features  incorporated  (compatibility,  observability,  controllability, 
partitioning,  etc.),  BIT  functional  design  and  BIT  interfaces  to  system 
BIT  and  to  external  test. 

10.3.3.3  For  each  item  to  be  included  in  the  Inherent  Testability  Assessment, 
recommended  weighting  factors  and  scoring  method  for  each  testability 
criteria  in  the  checklist. 

10.3.3.4  For  each  item  to  be  included  in  the  Inherent  Testability  Assessment,  a 
filled-in  checklist  and  the  calculated  innerent  testability. 

10.3.3.5  Description  of  methodologies,  models  and  tools  to  be  used  in  predicting 
built-in  test  fault  detection  and  fault  isolation  effectiveness. 

10.3.4  Detailed  Testability  Design  Analysis  (MIL-STO-2165 ,  Task  203) 

10.3.4.1  For  each  item  to  be  included  in  this  analysis,  a  definition  of  predominant 
failure  modes  to  be  tested,  a  prediction  of  built-in  test  fault  detection 
and  fault  isolation  effectiveness  and  identification  of  areas  which  require 
additional  testing. 

10.3.4.2  Prediction  of  built-in  test  fault  detection,  fault  isolation  and  false  alarm 
characteristics  at  the  system  level. 

10.3.4.3  Estimation  of  costs  associated  with  the  incorporation  of  built-in  test  and 
testability  features,  including  developmental  costs  and  recurring  costs. 
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DATA  ITEM  DESCRIPTION  (DRAFT) 


1.  title 

PM/FO/FL  MUDELING  REPORT  ' 


2.  IDENTIFICATION  NUMBER 
DI-ATTS-XXXA 


3.  DESCRIPTION  PURPOSE. 

3.1  To  describe  and  show  the  development  of  PM,  FD,  and  Fl  mathematical  model  reports 
to  be  used  for  making  numerical  PM,  FO,  and  Fl  apportionments  to  various  fucntions 
and  levels  of  nardware  and  software  throughout  an  item  and  to  evaluate  the  PM,  Fd, 
and  Fu  chaarcteristics  of  an  item  based  on  its  PM,  FO,  and  Fl  design  characteristics. 

4.  APPROVAL  DATE  5.  OFFICE  OF  PRIMARY  RESPONSIBILITY  6A.  OTIC  REQ  63.  GIDEP  REQ 

7.  applicatiun/interrelationship 

7.1  This  DID  satisfies  the  data  requirement  of  Dara  xxx  of  Appendix  x  or  me 

,  Statement  Of  work  of  contract 


8.  APPROVAL  LIMITATION 


9A.  applicable  FURMS 


9B.  AM SC  NUMBER 


io.  preparation  instructions 

10.1  The  PM,  Fu,  and  Fl  monels  shall  be  developed  in  accordance  with  Daragraon  y  of 
of  Appendix  X  of  the  Statement  of  Work  of  contract 
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DATA  ITEM  DESCRIPTION  (DRAFT) 

1.  TITLE 

PM/FU/FL  Allocation  Report 

2.  IDENTIFICATION  NUMBER 
DI-ATTS-XXXX3 

ION  PURPOSE 
ment  tne  quantitat 
oved  hardware  brea 
ents. 


4.  APPROVAL  DATE  5.  OFFICE  OF  PRIMARY  RESPONSIBILITY  6A.  OTIC  REQ  6B.  GIOEP  REQ 


7.  APPLICATIUN/INT£( 
7.1  This  DID  satisf 
'  Statement 


TIONSF 
he  da- 
ork  oi 


of  Append 


8.  APPROVAL  LIMITATION 


9A.  APPLICABLE  FORMS 


.  PREPARATION  INSTRUCTIONS 

.1  "The  PM,  FU,  and  Fl  allocations  snail  be  pe^mmed  in  accorc 
of  Appendix  X  of  tne  Statement  of  Work  of  contract 
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DATA  ITEM  DESCRIPTION  (DRAFT) 


1.  TITLE 

PM/i  Fl  Prediction  Report 


2.  IDENTIFICATION  NUMBER 
UI-ATTS-XXXXC 


3.  DESCRIPTION  PURPOSE 

3.1  To  document  the  quantitative  PM,  FD,  and  FL  preditions  made  by  the  contractor  and 
to  determine  whether  or  not  the  proposed  design  is  consistent  with  the  system  PM,  FD, 
and  FL  requirements. 


4.  APPROVAL  DATE  5.  OFFICE  OF  PRIMARY  RESPONSIBILITY  6A.  OTIC  REQ  6B.  UIDEP  RED 
7.  APPLICATION/INTERRELATIONSHIP 

7.1  This  OIU  satisfies  tne  data  requirement  of  pa^a  xxx  of  Appendix  x  of  tne 
.  Statement  Of  work  of  contract 


8.  APPKUVAL  LIMITATION 


9A.  APPLICABLE  FORMS 


95.  AM SC  NUMBER 


10.  PREPARATION  INSTRUCTIONS 

10.1  The  PM,  FU,  and  Fl  predictions  snail  be  performed  in  accordance  with  paraarapn 

y  of  Appendix  X  of  the  Statement  of  Work  of  contract  . 

10.2  The  PM,  FO,  and  Fl  prediction  reports  shall  contain  detains  such  as: 
a.  assumptions  used  in  the  prediction  process 

o.  identification  of  tne  prediction  procedure  used 
c.  prediction  results  to  the  appropriate  item  levels 
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FOR  TASKS 
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COiURaCT  OA  f  A  REQUIREMENTS  LIST 


Enclosure 


comtkac  r  data  h'-quirememts  list 
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INITIAL  DISTRIBUTION  LIST 


Addressee 


N>.  Of 
Ccpies 


ASST.  SEC.  NAV.  (SHPBLOG  8.  LOG.)  (D.  0.  Patterson,  L.  C.  Gills) 
SPACE  8.  NAVAL  WAREFARE  SYS  CMD  (SPAWAR-003-41 2 ,  D.  J.  Sellers  (3); 
SPAWAR-04-51 ,  CDR  Lopez  (3)) 

NAVSEASYSCOM  (SEA-06,  T.  Williams  (5);  PMS-393,  A.  D.  Lassiter  (3) 
PMS-396,  PMS-402,  PMS-414,  PMS-415  (D.  Hoffman), 

SEA-06Q1 1  (D.  G.  Bagley)) 

TRIDENT  CMD  8.  CTRL  SYS  MAINT  ACT 
NCSC  (Code  3110,  K.  Lane) 

Ryan  Computer  Systems  (K.  Hoffman) 

R.  M.  Vredenburg  L  Co.  (K.  Gardener) 


D.  Lassiter  (3) 


ft 
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